
Dynamic Open-Vocabulary 3D Scene Graphs for Long-term Language-Guided Mobile Manipulation
ACDC- Automated Creation of Digital Cousins for Robust Policy Learning
数字孪生(DT)作为现实世界非常精确的映射虽然可以用于高精度的训练但是生产DT资产过于繁琐且没有泛化性,不能做到zero-shot。
数字表亲(DC)通过比对模型特征,从模型库中选择类似的表亲模型,用于重建场景训练机械臂。让机械臂针对不同第一次见的场景具有泛化性。
(a)它减少了手动微调的需要,以保证一定的保真度,从而能够完全自动化地创建数字表亲,(b)它通过提供一组增强的场景来训练机器人策略,从而有助于更好地应对原始场景中的变化。
ACDC is our automated pipeline for generating fully interactive simulated scenes from a single RGB image, and is broken down into three steps:
(1) an extraction step, in which relevant object masks are extracted from the raw input image
(2) a matching step, in which we select digital cousins for individual objects extracted from the original scene
(3) a generation step, in which the selected digital cousins are post-processed and compiled together to form a fully-interactive, physically-plausible digital cousin scene.
参见 https://dingfen.github.io/2024/07/21/2024-7-21-ai_assist/
阿里云的大模型之前在搞wechat bot的时候接触过,这次再加上一个云函数的服务,就搞定了。
LLM:https://bailian.console.aliyun.com/?spm=a2c4g.11186623.0.0.2db33048PGTSE6#/app-center
云函数:https://fcnext.console.aliyun.com/applications?spm=5176.fcnext.0.0.65f378c8HmZpOP
在本地主要需要做的是在hexo中注入代码
添加文件scripts/injector.js
1 | hexo.extend.injector.register('body_end',` |
主要要更改的是window.CHATBOT_CONFIG.endpoint
, 改为触发器的公网访问地址
1 | hexo.extend.injector.register('body_end',` |
最终效果: