02 · Propose boxes
03 · Select a box
Drawer cropImage from stage 04
Candidate boxesFaster R-CNN
Choose a boxUCBUCB combines a candidate’s mean score with an exploration term that is larger for less-evaluated candidates.
Visual matchQwen2-VLThe VLM reads the crop and current target phrase, then returns a match score. The candidate’s running mean and count are updated.
ConsistencyHigh mean; low variationAfter the evaluation budget is spent, check candidates in score order. Slight brightness, rotation and scale changes should keep the mean above the threshold and the score range below its limit.
Accepted handle boxx: 200–320 · y: 90–120An accepted desk box becomes the input image for finding the drawer; the accepted drawer box becomes the input for finding its handle. Steps 02 and 03 repeat on each new image. In the schematic, the desk begins at (120, 140) in the scene, and the drawer begins at (90, 80) inside the desk. The drawer crop’s origin in the scene is therefore (210, 220). Keeping this offset makes it possible to search locally without losing the object’s original location.
Stop the chainIf the allowed retries accept no candidate, this level ends the chain. Earlier accepted regions remain in the output.
After budget: score order
Update mean + count
If rejected: try the next box
No accepted box after retries
From 01: current target = “black handle”
Accepted box → 04