LIDAR PERCEPTION · TRACKING

What LiDAR sees, and what it can't: lessons from tracking people at scale

Dr Akanda Ashraf · 2026 · all articles · back to profile

For around three years, first at Cron AI and now at Metrolla, I've worked on 3D LiDAR perception for tracking people and vehicles in live environments. LiDAR solves problems that cameras struggle with. It brings a few of its own too, and they rarely get discussed outside the teams working on them.

LiDAR measures distance directly. Each return is a range measurement, typically accurate to a few centimetres, so the position of a person or a vehicle comes from measurement, not inference. A single camera has to infer depth; stereo rigs measure it, but their error grows quickly with range. If the question is where something is, how fast it's moving and how close it is to something else, LiDAR answers it more directly. It is also an active sensor that brings its own light, so night, shadow and low sun matter far less than they do for a camera — heavy rain, fog, dust and very dark surfaces all cut the returns, but nothing eliminates them. And because it records geometry rather than images, with no faces and no video, it is far less identifying than a camera, which makes deployment in public spaces much simpler.

We call it 3D, but every point is measured at its own moment, and that fourth dimension is where most of the engineering lives. A scanning sensor doesn't capture the scene in an instant: a frame is swept out over up to around 100 ms, so different parts of the scene are recorded at different times, and anything moving is measured slightly out of step with its surroundings. Tracking is a time problem by definition — matching what you saw a moment ago with what you see now — and every challenge below comes back to time in some way.

The first challenge is identity in a crowd. A tracker working from LiDAR has shape, position and motion to go on, and very little appearance. When someone walks into a group, their points merge with their neighbours' or vanish behind them; when someone walks out, the tracker has to decide whether it's the same person. Camera trackers can fall back on appearance — LiDAR mostly can't — so track ID switches follow directly from what the sensor can observe. The work is to use every available cue (motion continuity, geometry, timing and scene context) so that switches stay rare and don't corrupt the counts and dwell times built on top of them.

The second is fusing several sensors. Most sites don't fit inside one sensor's field of view, so several are combined, which raises two problems. Compute is the first: several LiDARs produce a very large stream of points, and processing it in real time on edge hardware, rather than in a data centre, quickly becomes the constraint. Alignment is the second: a person walking at 1.4 m/s moves 14 cm in 100 ms, so an unaccounted timing offset of that size puts one person in two places across two sensors, and half a degree of angular misalignment becomes about 26 cm of error at 30 metres. Errors like these turn one person into two, or break a track just as someone crosses from one sensor's coverage into another's — so fragmented tracks and ID switches cluster at the handover zones: entrances, gates and corridors, which are usually the places that matter most.

The third is tracking when it gets busy. The arithmetic of matching detections to tracks can be kept under control; what grows in a dense scene is ambiguity. People walk close together, cross paths and occlude each other, so there are more near-ties to resolve, and the association methods that resolve them well cost far more to run than the simple ones. Busy periods are when accurate tracking matters most, and when it is most expensive.

Across these roles, a lot of my work has gone into these three problems: keeping multi-sensor detection and fusion inside an edge compute budget, keeping tracks consistent across handovers when timing and calibration are imperfect, and keeping tracking stable and affordable as crowds grow. The lesson that carries across all of it is to design around the sensor's physical limits from the start, not to treat them as edge cases once a system is deployed.

Cameras remain the right tool for plenty of jobs. For measuring occupancy, flow, safety and operations with precision and privacy, LiDAR is difficult to beat — provided you plan for identity, fusion and time from day one.