José Duarte

Polka Storage — Team lead & Rust developer

→ Responsible for setting the project course and keep management and the community up-to-date on development.

→ Research and development — kickstarted several parts of the project, such as, the first Rust library to support the CARv2 format; the parachain interfacing library; the end-to-end test suite; as well as several CI improvements.

Technologies: Rust, Polkadot, GitHub Actions & Projects

Full-stack development

→ Developed a complete backend with password-based authentication and refresh tokens.

→ Developed a proof of concept iOS app with Apple Maps integration.

→ Developed an automated data scraping pipeline to feed the app backend.

→ Deployed all infrastructure and performed system administration on a self-hosted Linux machine (including setting up VPN for 4 person team remote access).

Product research & development

→ Developed prototypes for early stage concepts.

Technologies: Swift (iOS), Python, SQLAlchemy, FastAPI, PostgreSQL, Docker, GitHub Actions

Feature development & maintenance

→ Maintained and developed features for the core product and its ecosystem.

CI/CD development

→ Migrated the self-hosted CI/CD pipeline from Azure Pipelines to GitHub Actions.

→ Maintained the new pipeline, reducing runtime (~25%) and adding new capabilities.

Technical writing

→ Wrote tutorials and technical guides (available at Actyx Docs).

Product research & development

→ Researched potential new products through the exploration of existing tools.

→ Developed application demos and proof-of-concepts.

Technologies: Rust, Typescript, Docker, Electron, Bash, Make, Swift (iOS), GitHub Actions, Azure Blob Storage

Distributed data processing system (akin to Prefect, Airflow and Dagster)

→ Developed and maintained the in-house distributed data processing system which processed the data (100s GB/day) behind the company's main product.

→ Developed and maintained the data access library supporting multiple backends — SQLite, PostgreSQL, Azure Blob Storage & File System.

→ Developed and maintained the distributed work library — providing an easy to use interface to the data science and research teams (~10 people).

Data ingestion pipeline

→ Developed (from scratch) and maintained the data pipeline responsible for ingesting data from all of the company's providers (100s GB/day).

→ The pipeline consisted of a custom integration between Prefect and dbt with incremental data processing.

→ Contributed fixes to upstream Prefect to unblock development progress.

→ Developed a data validation layer on top of the pipeline to ensure data quality.

Research & Development

→ Developed a "Pandas-to-SQL" compiler as a library — offering a Pandas like abstraction to query Snowflake.

→ Developed UDFs for key calculations — improving data processing runtime from hours to minutes.

Team Support

→ Provided feedback on several projects adjacent to the data pipelines.

→ Taught Python to non-technical members of the organization — from "zero" to EDA-capable contributors.

Technologies: Python, PostgreSQL, Snowflake, SQLite, Azure Blob Storage, RabbitMQ, Flask, GraphQL, Prefect, dbt

Contributor

→ Contributed with fixes and features to several projects, such as: Firefox, Pandas, SQLAlchemy, Poetry, Python LZ4, Prefect, GitHub Pull Requests for VSCode, Filecoin's Boost, & less_slow.py

Author

→ #[sealed] - a procedural macro implementing the sealed trait pattern. As of January 2025, it has ~8.5M downloads and is used in ClickHouse's official Rust client.

→ #[typestate] - a procedural macro providing an embedded typestate DSL for Rust.

Technologies: Rust, Python, git

Research & Development

→ Developed an embedded DSL for Rust, using procedural macros.

→ Published articles on the topic.

Technologies: Rust, LaTeX

Research & Development (IPMA partnership)

→ Developed a CUDA simulation model of the Fire Weather Index.

→ Studied two approaches for FWI prediction — ~500x500 grid with 1km² resolution over the period of June 2017 (date of the great wildfires of Pedrógão Grande).

Technologies: Python, C, C++, CUDA, QGIS

Team Management & Organization

→ Oversaw and organized the team participation in the competitions — within our first 6 months, we qualified for the CSAW 2020 finals.

CTF Player

→ Focused on Reversed Engineering, Cryptography, Web and Blockchain challenges.

→ Wrote several writeups on the challenges — you can find an incomplete list on GitHub.

Group Management

→ Oversaw and organized weekly open cybersecurity workshops.

→ Developed and presented several of the workshops.

Android Developer

→ Developed the first Portuguese Citizen Card Android SDK

→ Developed a demo app — able to read and write to the card, as well as sign data with the card's cryptographic keys.

Technologies: Kotlin, Android, Gradle

Mentoring

→ Provided CV building advice and guidance.

→ Provided interview advice — from the first contact to offer negotiation.

→ Leveled up junior engineers through code reviews, continuous feedback and support through career challenges.

Tutor

→ Tutored several students on a variety of computer science topics and languages — Data Structures, Computer Networks, Artificial Intellegence as well as C, Haskell, Python & Java.