2025.10.31组会
REVERIE数据集和DUET模型
- 现在都是还在前期的学习和探索阶段。 读这篇论文： REVERIE： Remote Embodied Visual Referring Expression in Real Indoor Environments （CVPR 2020）
REVERlE 发音 [ˈrevəri] DUET(Dual-scale Graph Transformer)
REVERIE 是 REVERIE： Remote Embodied Visual Referring Expression in Real Indoor Environments （CVPR 2020） 中引入的大规模远程目标接地数据集。
REVERIE 数据集是一个面向真实室内环境的远程县身视觉指代表达任务的基准，旨在评估具身智能体在真实室内环境中执行自然语言引导任务的能力。
与传统视觉语言导航 (VLN)任务不同，REVERIE 不仅要求智能体理解指令并完成导航，还需在抵达目标区域后识别并定位目标物体。
这一任务综合考察了智能体的语言理解、空间推理、视觉识别与跨模态融合能力，是典型的“导航+视觉指代表达”联合任务。该任务要求智能体:
- 根据自然语言指令在室内三维环境中导航
- 定位并识别目标物体。
| 1. Fold the towel in the bathroom with the fishing theme. 1. 将毛巾折叠在浴室里，以钓鱼为主题。 |
|---|
| 2. Enter the bedroom with the letter E over the bed and turn the light switch off. 2. 进入卧室，床上方有字母 E，关掉电灯开关。 |
| 3. Go to the blue family room and bring the framed picture of a person on a horse at the top left corner above the TV. 3. 前往蓝色家庭活动室，将电视上方左上角骑马的人的裱框照片带上。 |
| 4. Push in the bar chair, in the kitchen, by the oven. |
| 5. Windex the mirror above the sink, in the bedroom with the large, stone fireplace. |
| 6. Could you please dust the light above the toilet in the bathroom that is near the entry way? |
| 7. At the top of the stairs, the first set of potted flowers in front of the stairs need to be dusted off. |
| 8. To the right at the end of the hall, where the large blue table foot stool is, there is a mirror that needs to be wiped. |
| 9. Go to the hallway area where there are three pictures side by side and get me the one on the right. |
| 10. There is a bottle in the office alcove next to the piano. It is on the shelf above the sink on the extreme right. Please bring it here. |
评价准则：
REVERIE任务采用Remote Grounding Success (RGS) rate作为主要的评价准则，它是成功找到目标物体的任务数与总任务数的比值。而是否成功找到物体有两种评价方法：
1）如果Pointer从模拟器提供的若干候选物体中进行选择，则选出正确的物体为成功。
2）如果Pointer从物体检测器提供的候选物体中进行选择，则预测输出的边框与真实边框的交并比不小于0.5视为成功。