Robotics & perception · IHCI 2026 · Paper 105
Robot–Elevator Interaction
Recognising elevator controls and carrying out an operator-requested button press.
Next Generation Internet of Everything Laboratory · Electrical and Electronic Engineering
University of Nottingham Ningbo China
Accepted for oral presentation at IHCI 2026. Paper available as the September 2026 author’s version.
404 / 475 requests correct, derived from 112 saved RGB images across six buildings.
Macro-F1 on 379 annotated Open / Close / Other button crops.
Operator-initiated button activations; qualitative examples, without a repeated-trial success rate.
Project overview
Working with an existing lift panel avoids a manufacturer-specific interface, but connects several different tasks: finding buttons, interpreting labels, choosing the requested target and mapping it into the arm’s coordinate frame.
The prototype runs on a Jetson AGX Orin with an RGB-D camera, a wrist camera and a robot arm. An operator positions the cart and requests each press; the cart stays stationary during the demonstrations.
Results and scope
Perception benchmarks, runtime observations and physical demonstrations answer different questions. The saved-image target-selection result does not measure an end-to-end live pressing success rate.
| Measure | Result | Setting |
|---|---|---|
| Requested-target selection | 404 / 475 · 85.1% | 112 saved RGB images from six buildings |
| Open / Close / Other classification | Macro-F1 0.9963 | 379 annotated button crops |
| Redundant semantic requests filtered | 3,532 / 3,540 · 99.8% | One 207-second runtime observation |
| Physical button activation | Two qualitative demonstrations | Hall down-call and cabin floor 2 |
99.8% is a request-filtering rate. It is not recognition accuracy or a button-press success rate. The two activations do not establish repeatability across elevators; press force and positioning accuracy were not quantified.
Both recordings show physical contact, button illumination and withdrawal. The next step is repeated live perception-to-press evaluation, with clearer outcome checks and recovery across varied panels.
Materials
- PaperPDF · 12 pages · 0.5 MB
September 2026 web edition. All 12 paper pages are retained.
- IHCI 2026 presentationPDF · 10 pages · 3.6 MB
10-slide presentation covering the platform, recognition results and two physical demonstrations. Video areas use still frames; the recordings are playable above.
- FoSE research showcase posterPDF · 1 page · 1.5 MB
Original A1 layout and team credits; student numbers removed from this public copy.
The full-resolution paper is also available. Sharing of the author’s version is subject to Springer Nature’s terms .
Open full poster
Research record
Robot–Elevator Interaction: From Physical Panel Recognition to Activation
IHCI 2026 · Paper 105 · Accepted for oral presentation.
Official programme: 29 October 2026, 15:00–16:00 · SS4 · IEB 124 · Ningbo (UTC+8). Proceedings publication, DOI and indexing remain unconfirmed.
Paper abstract
Existing robot-accessible elevator interfaces require manufacturer-specific retrofits which are costly and difficult to scale. Direct physical interaction with existing elevator panels offers a promising infrastructure-independent alternative, but requires robots to reliably perceive, interpret, and actuate controls despite variations in panel layouts, lighting conditions, reflections, and labeling conventions. We present an edge-based prototype system with an end-to-end perception-to-action pipeline that integrates task-specific button detection, specialized target recognition, vision-language semantic understanding, confidence-aware rejection, semantic-state maintenance, asynchronous processing, and physical button actuation. We evaluate perception performance, static-image target selection, runtime behaviour, and physical actuation through two qualitative button-activation demonstrations. Across 475 target-selection requests derived from 112 static RGB images spanning six buildings, the system selected the intended button in 404 cases, achieving an accuracy of 85.1%. On a separate benchmark of 379 human-annotated Open/Close/Other button crops, the HOG+SVM classifier achieved a macro-F1 score of 0.9963. During a 207-s runtime evaluation, the system filtered 99.8% of redundant candidate-level semantic requests, with no observed label switching, flicker, crashes, or camera interruptions. In both operator-initiated physical button-activation demonstrations, the robotic arm activated the selected button, withdrew, and returned to its standby state. These results provide practical insights and design considerations for the development of robust, edge-based, offline, and real-time robot–elevator interaction systems.