RoboDE is a robotics IDE. A Planner LLM turns an instruction into robot tasks, a Task LLM turns each task into robot commands, and every motion is checked before it runs in NVIDIA Isaac Sim on a cloud GPU.
Recorded on the RoboDE server running entirely on a Google Cloud GPU VM, connected live to Isaac Sim.
The two roles can use different models and providers. Here the Planner is Nemotron 3.5 Bolt on the GPU VM and the Task LLM is Nemotron 3 Nano 30B. Each plan, and each Business App command, can override them.
From the instruction “move the apple from Table1 to Table2”, Nemotron 3.5 Bolt produced nine ordered tasks for arm_01, each with its dependencies. Tasks stay editable before anything is converted or run.
The Environment workspace connects to the Isaac Sim worker on the GPU VM, reads the live scene and resets it. A native Isaac camera recording starts at 1280×720, 10 fps.
Validation compiles the plan into trajectories and checks reach, joint limits and collisions. Run sends it to Isaac Sim; the 3D view shows execution feedback as each task completes.
Each run reopens its exact compiled revision. Postconditions are verified from the state Isaac Sim reports, and the log lists every task with a timestamp.
Joint sliders stream targets to the GPU scene, and the measured joint values come back from Isaac's articulation. In the recording, each move was confirmed against that feedback before the next one started.
Safety check in actionOne scripted move would have lowered arm_02 into a table. RoboDE refused it with “Link intersects table at sample 30” and stopped that control session before anything moved.
Task LLM output is validated against the robot's command catalog, normalized, and repaired once if the JSON is malformed. Anything else is rejected with a specific error.
Commands become trajectories through inverse kinematics, with analytic and serial_chain_dls_v1 solvers.
Joint limits, robot-to-robot, table and object collisions are checked on every sample of every trajectory.
After a run, the result is verified from observed simulator state, and each compiled revision is immutable.
Each model ran once per test, with pass or fail checks. These are functional checks of RoboDE's planning flow, not general model rankings.
All four rejected unreachable poses, unknown actuators and joint-limit violations. All four failed the three adapter-generation tests. Only Bolt passed the 30 cm object lift, and it was the most reliable Planner on the live Isaac project.
| Test | Phi-4 14B | Mini 4B | Nano 4B |
|---|---|---|---|
| Full plans | 4/6 | 0/6 | 0/6 |
| Task conversion | 2/2 | 1/2 | 1/2 |
| Adapter from docs | 1/1 | 1/1 | 0/1 |
| Adapter time | 38.6 s | 11.7 s | 34.9 s |
Phi-4 won on planning. It is about 3.5 times larger than the 4B Nemotrons. Bolt was not tested against Phi-4.
| Model | Furthest stage reached | What happened |
|---|---|---|
| Nemotron 3 Ultra 550B | Validation | 12 tasks, converted to commands; stopped on a tool orientation the demo arm cannot reach (RoboDE now handles this). |
| Nemotron 3.5 Super VL | Validation | 7 tasks, converted; final placement check failed. |
| Nemotron 3 Super 120B | Conversion | Good 6–12 task plans; Task LLM step left move targets incomplete. |
| Nemotron 3.5 Lightning | Conversion | Short 4-task plans; token limits and conversion errors. |
| Nemotron 3 Nano 30B | Planning | Plans failed; used as Task LLM in the demo because of remaining credit. |
No model produced a fully valid plan end to end without help. Many hosted runs were cut short by API credit and free-tier limits, and the hosted runs used a test scene, not the live Isaac project. Every failure was traced to a fix in RoboDE's planning flow.
The standard edition. Desktop app with local calculations, or a team server with the browser IDE and Business App.
NVIDIA accents and a first-boot setup that defaults to Isaac Sim on a Nebius GPU VM and Nemotron on Nebius Token Factory. Other simulators and models stay available.
A phone-friendly companion: named workspace assets, natural-language and voice commands with model choice, and room photos.