- Mon 16 January 2023
- Projects
- Mark Snidal
Hello there! I'm a software and data platform engineer, currently based in San Francisco and working on agentic/generative AI product at Connectly, where I'm designing and building distributed systems, core data infrastructure, and other backend work to support our ecommerce AI sales agent product. I'm interested in scaling software, data, and machine learning systems and infrastructure, and in particular in the application of language models to all kinds of problems. I have recently contributed to a number of open-source projects:
- terraform-provider-pinecone (go): Added support for metadata field indexing to the Terraform provider for the Pinecone vector search database
- dspy (python): Fixed and overhauled the Pinecone retrieval module for dspy, a LLM prompting framework project from the Stanford NLP group
- fastapi-profiler (python): Implemented standard
.prof
profiling file support for a FastAPI middleware profiling tool - gpt-review (python): Added support for calling the OpenAI API directly where previously only Azure was implemented on this GitHub pull request GPT generation tool by Microsoft
- feast (python): Fixed some errors relating to PostgreSQL types for the Feast ML feature store (mainly so I could use it at work, lol)
I have also authored a handful of projects, including:
- Stitch ETL integration for the Dagster data platform
- attercop, a command-line GPT shell tool
- precinct, a SQL query AI copilot and associated vscode extension
I've worked in startups in Canada and the US, after doing some internships in the video game and aerospace industry and getting my software engineering degree from Concordia University in Montreal. Prior to joining Connectly in 2023 I have worked at the following companies:
- Industrious (2021-2023): Led the development of a new data warehouse and platform consisting of all the revenue modelling and forecasting for the company and hundreds of new data models models and a data orchestration platform to support operations and analytics
- Breather (2020-2021): Managed ingestion, cloud data infrastructure, our dbt warehouse, Luigi ETL pipelines and delivered a new price occupancy machine learning model. Breather was acquired by Industrious in 2021
- Dataperformers (2018-2020): Joined a small bootstrapped Montreal-based team of machine learning researchers as a cloud architect and data engineer out of school, applying my software engineering background to help deliver machine learning projects for various local clients
Blog
- Sun 08 January 2023
- Projects
- Mark Snidal
Attercop: In-line shell command generations with natural language
- Sat 07 January 2023
- Housekeeping
- Mark Snidal