Jetson Orin Nano Super Robotic Arm with ROS 2 Vision-Guided Picking Development

Phase 01 · Hardware

Hardware and System Architecture

The reference cell contains a six-axis industrial or collaborative robotic arm together with its respective controller, changeable end effectors, and an RGB-D camera mounted vertically above the picking area. One of the cameras is fixed so that the tray can be easily scanned, and an optional camera mounted on the wrist can help examine difficult picks or verify placement.

Camera choice depends on working distance, field of view, depth accuracy, reflective surfaces, and the available mounting structure.

The NVIDIA Jetson Orin Nano Super runs the ROS 2 perception and task-coordination software locally.

NVIDIA describes the Super performance improvement as a software-enabled upgrade to the Orin Nano developer kit; the available compute does not by itself establish how many camera streams or picks per minute a particular application can support. Those figures must be measured with the chosen camera, model, planner, and robot.

One distinct controller is used to handle arm movements. There is a robotic gripper controller that tells us if it's opened, closed, or else states that p.

A PLC or digital I/O takes care of cell-ready signals, detection of being present on the tray, stack lights, and other machine operations necessary. Hardwired safety risks are treated separately from the application running on the Jetson.

Phase 02 · Software

ROS 2 Software Architecture

The graph of ROS 2 divides the process of vision acquisition of a camera, inferencing, selection of a target, motion control, and regulation of robots. Nodes of camera practice use the process of publishing the colour images with time stamps, depth images, and the details of the calibration process.

The node of perception publishes the information about objects that have been identified together with their characteristics such as type, verification level, location on the image, and an estimated three-dimensional position. The function of a task coordinator involves choosing a target, requesting for a motion plan, overseeing the process of grasping, and keeping appropriate records.

The design of the robots provides the details such as links, joints, programming the configuration of tools, limitations of joints, and the configuration of some special positions. Transform frames link elements such as a camera, the bottom part of a robot, tool center point, and tray. The operation of the Controller Interface allows to receive the data about the position of joints of a robot and to execute various commands.

This separation makes a critical fault visible: if the camera pose drifts, the team can inspect calibration rather than repeatedly retraining the detector. ROS 2's hardware-interface model provides the relevant command and state concepts for controller integration.

A state machine governs the cycle: confirm the cell is ready, acquire a fresh image, select a target, plan an approach, move above the part, descend, close the gripper, verify acquisition, retreat, move to the destination, release, and confirm completion.

Each transition has a timeout and a defined fault response. The system never assumes that a planned pick was physically successful merely because the motion command completed.

Phase 03 · Perception

Depth Perception and Calibration

After calibrating the intrinsic parameters of the camera, the camera is accepted into the robot control system. When the camera is mounted overhead, calibration determines how the camera is linked to the robot's base.

However, when the camera installation is movable (eye-in-hand application), calibration needs to identify how the camera is related to the tool being operated. It is critical to ensure that calibration is tested every time the camera is altered or the tools being used have been modified.

The detector or segmenter identifies candidate objects in the colour image. Depth pixels associated with a candidate provide a point cloud or surface estimate.

The pipeline rejects invalid measurements, tray-wall artefacts, and points outside the permitted pickup area. Depending on the part, it estimates a centre and orientation from a segmented contour, a fitted geometric primitive, or a trained grasp proposal.

Depth sensing has limits. Glossy metal, transparent packaging, dark surfaces, occlusion, and closely stacked parts can produce unreliable geometry.

For these items, practical options include changed lighting, a different depth technology, multiple viewpoints, a known tray fixture, or declining the pick and requesting operator review. A reliable system should recognise when it lacks a usable grasp estimate.

Phase 04 · Motion

Grasp Selection and Motion Planning

Grasp candidates are evaluated according to certain parameters like their confidence in the object position, surface area available for contact, width of the robot's hand or suction mechanism, distance from the walls of the tray, accessibility, and a probable requirement for wrist rotation. The grasp candidate will then be expressed in terms of the robot's coordinate frame and will be performed together with the approach pose, and the poses for picking and pulling away.

MoveIt 2 keeps a planning scene to contain the arm of the robot, the tools, tray, immobilizers, and other elements of collision. The motion planner compares all possible motions to the built model and specifications of the robot.

The software should check the results of the operation instead of relying on the planner's signal as if the operation was successful. Collision checking in the planning scene is described in the MoveIt 2 planning scene documentation.

For a crowded tray, the first reachable object may not be the best one. Picking a topmost item can expose another part and reduce collision risk on the next cycle. Conversely, a high-confidence detection near a tray wall may be rejected because the gripper cannot approach it safely. These decisions belong in the task logic, with rejection reasons recorded for later improvement.

Phase 05 · Validation

Integration, Failures, and Field Validation

The status indicators of the supervisor are available for the overall production process, including ready, busy, completed and fault statuses. The operator console may include the camera's view, target chosen, robot's status, last fault and batch count.

It will permit designated personnel to put a hold on a certain cell, acknowledge a fault and initiate the process from a certain state with regards to safety systems.

Verification starts with filming and simulating the movement, moving into slow runs and controlled belt picking.

The testing set consists of empty racks, things that overlap, undesired items, taking into account received vision with insufficient depth, etc.

We will define the quality of detection success as well as the speed of gripper locking, placement accuracy, recovery quality and full cycle duration.

A pilot should run across the actual range of trays, shifts, lighting conditions, and part variations. Its outcome determines whether the next improvement is a model update, a revised grasp strategy, a better fixture, or a hardware change.

No throughput or reliability claim should be published until measured against an agreed test protocol on the physical cell.