Data Engineer

uPlanner is an AI-powered B2B SaaS for higher education (founded 2012, HQ Miami, 90+ specialists, offices across 8 countries). It serves 30+ institutions including Universidad de los Andes, PUC Chile, and Aliat Universities Mexico — at scale, 20 Peruvian public universities simultaneously covering 350,000 students. The company was named one of the Americas' Fastest Growing Companies by the Financial Times in 2023 and holds ISO/IEC 27001 certification. Products span the full institutional lifecycle: uForecast (demand), uPlanning (scheduling), uBooking (events), uAssessment (curriculum), uEngagement (retention), uClass (attendance/grades), uExperience (mobile app), and uSuite (integrated platform). Solutions are frequently customized per client — every product can vary in implementation detail, so engineers must maintain deep awareness of client-specific configurations across multiple simultaneous deployments.

As a Data Engineer on the Data Services team (3–6 people, fully remote), responsibilities spanned the full lifecycle of data pipelines: design, development, maintenance, documentation, and client-facing support. Operated simultaneously as a pipeline developer, integration engineer, cloud-migration contributor, internal-tooling builder, and technical representative in international client meetings. Carried significant autonomy — owning end-to-end integrations, leading client meetings, and independently driving projects from requirements through delivery.

ETL/ELT pipeline ownership. The core ETL system was built on Pentaho Data Integration, organized into 6 sequential steps per client per run (SFTP file discovery → format validation → staging load to MariaDB → cross-referencing into product DB → algorithm execution → report generation, plus a MongoDB key-value step for mobile data). Python scripts were layered throughout the pipeline to handle client-specific transformations, validations, and business logic across multiple simultaneous deployments.

D2L Brightspace LMS integration suite. Owned all integrations with D2L Brightspace REST API — uPlanner's primary external data source, used across 5 international clients in Colombia, Mexico, Argentina, France, and others. Built grades processing, attendance tracking, course/curriculum fetching, and curriculum upload automation, all following a uPlanner-MySQL → D2L-API → cross-reference → business-logic → output pattern. Handled token-based auth (credentials in DB, access tokens refreshed per session), pagination for endpoints returning tens of thousands of records, and async export-job polling for D2L endpoints that trigger background jobs rather than returning data immediately.

Curriculum upload automation (Colombia client). A custom Python integration that fetched uPlanner's generated curricula from MySQL, cross-referenced each against D2L courses via the Brightspace API, and programmatically uploaded curriculum files to D2L as they became ready. Replaced a manual teacher workflow consuming 250+ hours per semester.

Automated error logging. Designed and implemented systematic error capture, categorization, and surfacing across ETL transformation and validation stages — reducing diagnostic and resolution time on pipeline issues by 43%.

Pentaho → Azure Synapse migration. Initiated as greenfield: designed and built the first complete ETL pipeline from scratch on Synapse to evaluate the tooling, establish development patterns, and measure performance. Used findings to define the migration strategy for legacy pipelines, then began migrating in batches. Synapse components used: Pipelines, Notebooks (Python/PySpark), Spark Pools, SQL Pools, Data Lake storage. Improved integration efficiency by 35%. Migration was underway at time of departure (first new pipeline built; 2 legacy pipelines successfully migrated).

Internal Streamlit platform. Designed and developed a Streamlit web platform consolidating 20+ internal tools used across Data Engineering, Product, Customer Success, and Data Science. After two years of repeatedly solving the same categories of CS-reported issues, the recurring query patterns were compiled into purpose-built, form-driven tools — CS staff went from waiting up to 2 hours per issue (no SQL knowledge, blocked on engineer availability) to under 2 minutes via instant self-service. Tools included file transformation/parsing utilities, recurrent DB query runners with friendly UIs, DB job management, environment/client config comparison, and API testing interfaces.

Data model restructuring. Led cross-functional meetings with technical, product, and customer service teams to restructure the data model and realign product requirements, identifying misalignments between internal data structure and client onboarding needs. Reduced initial customer data delivery time by 20%.

Client-facing work. Served as the technical face of the Data Services team in international client meetings — requirements gathering, feasibility evaluation, project follow-ups, and live-deployment issue resolution. Frequently led meetings directly across Colombia, Mexico, Argentina, France, and other countries.

Infrastructure & DevOps. uPlanner runs a multi-tenant database architecture on MySQL where each client is provisioned with isolated development, QA, and production instances — so every pipeline, integration, and tooling change had to be environment-aware and validated against QA before promotion to live institutional data. The team standardized on the Atlassian Suite for the full delivery lifecycle: work was tracked as Jira tickets, version-controlled in Bitbucket, and shipped through Bamboo's automated deployment pipelines across the dev → QA → prod environments.

Mentorship & documentation. Mentored 2–3 engineers across the team lifecycle, both during onboarding and as part of formal growth plans. Maintained technical documentation for all owned developments (pipelines, integrations, automations, tools) as a team-wide standard.