"Smile because it happened" — Dr.
Seuss The Setup Earlier this year, I took on a short-term trial role with an early-stage robotics startup.
The premise was straightforward: help with data collection, annotation, and evaluation workflows—essentially the backbone of any modern robotics or embodied-AI system.
The trial didn't work out long-term.
I was let go after about two months — a decision that, honestly, came down in part to my bandwidth as a student.
Balancing a full course load with a startup trial was harder than I anticipated.
But that's not the story I want to tell.
What I do want to share are the technical lessons I took away — lessons about building robust data pipelines, about the gap between theory and practice, and about what I'd do differently next time.
These aren't company secrets.
They're about the general engineering challenges that anyone working with robotics data pipelines will encounter — challenges I'd read about in papers but hadn't truly internalized until I was standing in front of them.
1.
The Data Pipeline Shape Is Universal—But the Details Aren't If you've spent any time in ML or robotics, you've seen this described: Data Collection → Annotation → Evaluation It's a standard three-stage pipeline.
Industry vendors describe it explicitly in their robotics content.
Academic projects model this structure.
It's the field's shared vocabulary.
Companies such as Scale AI and Toloka use similar industry workflows involving data collection, annotation, and evaluation.
What isn't shared are the specifics: the sensor setup, the calibration procedures, the annotation rubric, and the evaluation metrics.
Those are where a company's IP lives.
The pipeline shape?
That's just the map.
And the map is public.
What I'd do differently: Simulate before you collect.
Data collection is expensive — in time, hardware wear, and cognitive load on operators.
Before running a full session, run a feasibility study with a small batch.
Verify your sync and capture scripts.
I assumed this was standard practice.
It wasn't.
The cost of that assumption showed up as rework.
2.
Annotation Granularity: More Isn't Always Better There's a natural temptation when designing an annotation schema: capture everything.
Every possible label, every edge case, every attribute you might want later.
This is a trap.
Over-granular annotation creates: Higher operator cognitive load → slower throughput, more errors Greater inconsistency → more labels means more disagreement Fragile evaluation → you're evaluating against things you might not need The better approach?
Start with the minimal viable schema that answers your core research question.
Add complexity only when the data tells you it's necessary.
A concrete heuristic: If you're designing labels for a demonstration-collection task, ask yourself: "Will I actually use this attribute to decide if the demonstration is successful?" If the answer isn't an immediate yes, drop it.
Retrofitting granularity is easier than cleaning up inconsistent over-labeling.
3.
Automation Still Needs Oversight—Especially in Evaluation Automation is seductive.
The idea that you can "just set up an automated evaluation pipeline and let it run" is appealing, especially in a startup where resources are tight.
I learned the hard way that automated evaluation is not a replacement for human judgment.
It's a filter.
Automated checks catch obvious failures (missing data, format errors).
They don't catch subtle issues (context-dependent errors, edge cases).
They create a false sense of confidence without regular spot-checks.
The workflow that actually works: Run your automated eval, spot-check a sample of what passes, find patterns in the failures, add those to your automated checks, and repeat.
It's the "human-in-the-loop" pattern.
Easy to forget when you're moving fast.
What I'd Tell My Past Self If I could go back to Day 1 of that trial, here is exactly what I'd say: Ask for explicit success criteria upfront. "Good work" means diff