3D Vision Robot Guidance: Pick Point Identification Using CAD Matching

Oculus Vision provides specialist machine vision support to system integrators and machine builders delivering complex automation projects. This page describes our approach to a common but technically demanding challenge: achieving precise, repeatable robotic pick operations using 3D vision and CAD matching.

The Challenge: Repeatable Picking with a Stereo Camera

Our client required a robust vision solution to identify and pick components from a scene using a robot, with high accuracy and repeatability. The application was machine tending — where a component must be presented to a machine in a precise and consistent orientation every cycle. Standard pick point identification approaches were insufficient for the required level of precision.

Why Standard Approaches Fall Short

Many 3D vision systems can automatically identify suitable pick points by locating an unobstructed surface area on an object. This is fast and requires no prior knowledge of the object's shape, making it suitable for general pick-and-place tasks.

However, because the system is finding any valid accessible point, the exact pick location is not guaranteed to be consistent between cycles. If the object shifts slightly or is partially obscured, a different pick point may be selected. For precision applications where consistent part orientation is critical, this variability is not acceptable.

The Oculus Vision Approach

We selected the Balluff rc_visard stereo smart camera to capture accurate 3D point cloud data of the scene. Rather than relying solely on the camera's onboard pick point software, we exported the point cloud data to a separate processing environment running MVTec Halcon — a leading machine vision software platform in which we have deep expertise.

Prior to deployment, we used Halcon to generate surface models of the target components from their CAD data, and manually defined a specific, repeatable pick point on the virtual model. This pick point is used consistently on every cycle, regardless of how the component is presented.

At runtime, the rc_visard captures the 3D point cloud and passes it to the Halcon application running on an industrial PC. Halcon locates the component precisely within the scene — determining not just XYZ position but also yaw, pitch, and roll orientation. This allows the pre-defined CAD pick point to be accurately translated into real-world coordinates, ensuring the robot picks from exactly the same location every time.

Note: The images shown below use cameras as demonstration objects in place of the actual production components, which remain confidential to the client.

Why This Matters

The combination of the Balluff rc_visard and MVTec Halcon CAD matching delivers a level of precision and repeatability that standard pick point approaches cannot match. By taking responsibility for the full vision scope — hardware selection, system architecture, software development, and commissioning — Oculus Vision was able to deliver a solution that met the client's rigorous requirements for accuracy and production reliability.

This approach is applicable wherever robotic handling requires consistent, orientation-aware pick operations on known components — including machine tending, assembly, and precision handling applications.

For more on hand-eye calibration in robot vision applications, see our technical note.