Python-Unity Network
Robotic Arm Digital Twin(DT)
Posted Updated Notea minute read (About 192 words)

Robotic Arm Digital Twin(DT)

Robotic Arm Digital Twin(DT)

机械臂 DT 需要的功能：

  • 有和现实机械臂相似的模型外观
  • 有碰撞箱
  • 有基本的关节旋转控制脚本
  • 接受路径规划单元传来的控制数据

模型外观

三个模型资产，分别从

导入。

碰撞箱

主要由 capsule collider 构成。整个 DT 的 layer 属于 RobotArm

关节控制脚本

挂载在 robotarm 物体上，用于控制各个关节旋转的角度

其中 RotateDir 为一次性配置，Joint 会自动检测进行配置

控制时只需要更改 Rotate 的值即可（runtime）

IK Path Planning
Posted Updated Note3 minutes read (About 425 words)

IK Path Planning

IK Path Planning

IK Solver

用于将关键点数据转化为关节动作（各关节旋转角度）

希望机械臂的末端始终竖直。方便夹取。

Unity IK: https://github.com/2000222/Robotic-Arm-IK-in-Unity

自定义自由度，非常好用，基于关节梯度迭代的方法

由这个 IK 算法负责机械臂前三个 joint 的旋转，确保 j4 的 anchor（红点）处于物体后上方的特定位置。

j4 位置的计算方法：

1
2
3
4
5
6
7
8
9
10
11
public Vector3 GetPositionForJ4(Vector3 pos)
    {
        Vector3 forward = pos - Joints[0].transform.position;
        forward.y = 0;
        float angle = Mathf.Acos(0.1148f / forward.magnitude);
        Vector3 left = Quaternion.AngleAxis(-angle * 180f/Mathf.PI, Vector3.up) * forward;
        left = left.normalized * 0.1148f;
        Vector3 offset = forward - left;
        offset = offset.normalized * 0.1175f;
        return pos + Vector3.up * 0.2446f - offset;
    }

之后由向量计算得出 j4 应该旋转的角度，以确保末端竖直。

j5 不旋转，j6 的旋转角度由物体朝向决定。

Router

Route

一条路径由若干个关键点构成，在构建路径时只需输入若干 turning point，就会自行生成路径中的关键点。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
public class KeyPoint
{
    public Vector3 pos = Vector3.zero;
    public bool grab = false;
    public bool wait = false;
    public KeyPoint(Vector3 pos, bool grab = false, bool wait = false)
    {
        this.pos = pos;
        this.grab = grab;
        this.wait = wait;
    }
}

public class Route
{
    public List<KeyPoint> routes = new List<KeyPoint>();
    public Route(List<KeyPoint> turningPoints)
    {
        if (turningPoints.Count < 2)
        {
            return;
        }
        for (int i = 1; i < turningPoints.Count; i++)
        {
            KeyPoint start = turningPoints[i-1];
            KeyPoint end = turningPoints[i];
            float dist = (start.pos - end.pos).magnitude;
            int sigment = (int)(dist * 20f);
            for (int j = 0; j <= sigment; j++)
            {
                routes.Add(new KeyPoint(start.pos + ((float)j) / sigment * (end.pos - start.pos), start.grab));
            }
        }
    }
}

IK 只需要获取关键点的坐标信息就可以生成各个关节旋转的数据。

1
InverseKinematics(GetPositionForJ4(position));
JustWork 参考文档
Posted Updated Notea few seconds read (About 78 words)

JustWork 参考文档

Unity 本体

新的Input System:https://gamedevbeginner.com/input-in-unity-made-easy-complete-guide-to-the-new-system/#how_to_manage_input

物理引擎

目前的想法是将unity的physx和mujoco结合在一起，简化physx的对于相对运动很少的刚体间的物理解算。
Mujoco:https://mujoco.readthedocs.io/en/latest/unity.html

Game Principle of <JustWork>
Posted Updated Note2 minutes read (About 363 words)

Game Principle of <JustWork>

参考

Besiege

核心功能

机械建造

需要更高的建造自由度：例如刚体零件的顶点自定义，3轴移动，3轴旋转，3轴缩放。
可视化内容更丰富：例如碰撞箱，连接点。

物理优化

目前还是使用phsX引擎，考虑一下GPU加速这块功能。
零件之间的连接考虑使用joint以外的方式，特别是某些刚性连接。

逻辑电路体系

参考我写的BesiegeModern Mod

零件材质自定义

{金属，木，玻璃}
需要自己写shader material实现该功能。

次要功能

更好的机械破坏系统

譬如撞击变形，高速击穿之类的
很难
在已有父刚体在运行中，更改碰撞箱布置是非常耗费性能的。

场景建造

自定义场景元素

教程系统

参考mc机械动力模组的“思索”
animation制作

故事模式

起部分教程作用，添加沉浸感，吸引一些非硬核玩家

三渲二

难，可能对电脑性能有较高要求。

目前需要的技术栈

unity shader
unity animation
美术相关工具
物理引擎Game Physics Engine Development
unity URP

ONI Modding
Posted Updated Note3 minutes read (About 490 words)

ONI Modding

Basic

Useful tools

  • Dnspy: decompile the runtime library of ONI
  • Visual Studio (though it’s too fat)
  • Poedit: read the .po file for zh-en translation

Hello world

Apply patches on the Initialize() method in Db class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
public class Patches
    {
        [HarmonyPatch(typeof(Db))]
        [HarmonyPatch("Initialize")]
        public class Db_Initialize_Patch
        {
            public static void Prefix()
            {
                Debug.Log("Hello! Welcome to CuppyUniverse! [before Db.Initialize]");
            }

            public static void Postfix()
            {
                Debug.Log("CuppyUniverse initialized! [after Db.Initialize]");
            }
        }
        [HarmonyPatch(typeof(ElectrolyzerConfig))]
        [HarmonyPatch("CreateBuildingDef")]
        public class ElectrolyzerConfig_Patch
        {
            public static void Postfix(ref BuildingDef __result)
            {
                __result.EnergyConsumptionWhenActive = 114f;
            }
        }
    }

Add new building

Take gold washer (just like water purifier but output small amount of gold) as an example.

  1. Find the source code for water purifier.
    i.e.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    using System;
    using TUNING;
    using UnityEngine;

    // Token: 0x0200039A RID: 922
    public class WaterPurifierConfig : IBuildingConfig
    {
    	// Token: 0x06001314 RID: 4884 RVA: 0x000665C4 File Offset: 0x000647C4
    	public override BuildingDef CreateBuildingDef()
    	{
    		string id = "WaterPurifier";
    		int width = 4;
    		int height = 3;
    		string anim = "waterpurifier_kanim";
    		int hitpoints = 100;
    		float construction_time = 30f;
    		float[] tier = BUILDINGS.CONSTRUCTION_MASS_KG.TIER3;
    		string[] all_METALS = MATERIALS.ALL_METALS;
    		float melting_point = 800f;
    		BuildLocationRule build_location_rule = BuildLocationRule.OnFloor;
    		EffectorValues tier2 = NOISE_POLLUTION.NOISY.TIER3;
    		BuildingDef buildingDef = BuildingTemplates.CreateBuildingDef(id, width, height, anim, hitpoints, construction_time, tier, all_METALS, melting_point, build_location_rule, BUILDINGS.DECOR.PENALTY.TIER2, tier2, 0.2f);
    		...
    		return buildingDef;
    	}

    	// Token: 0x06001315 RID: 4885 RVA: 0x000666A8 File Offset: 0x000648A8
    	public override void ConfigureBuildingTemplate(GameObject go, Tag prefab_tag)
    	{
    		...
    	}

    	// Token: 0x06001316 RID: 4886 RVA: 0x0006688B File Offset: 0x00064A8B
    	public override void DoPostConfigureComplete(GameObject go)
    	{
    		...
    	}

    	// Token: 0x04000A88 RID: 2696
    	public const string ID = "WaterPurifier";

    	// Token: 0x04000A89 RID: 2697
    	private const float FILTER_INPUT_RATE = 1f;

    	// Token: 0x04000A8A RID: 2698
    	private const float DIRTY_WATER_INPUT_RATE = 5f;

    	// Token: 0x04000A8B RID: 2699
    	private const float FILTER_CAPACITY = 1200f;

    	// Token: 0x04000A8C RID: 2700
    	private const float USED_FILTER_OUTPUT_RATE = 0.2f;

    	// Token: 0x04000A8D RID: 2701
    	private const float CLEAN_WATER_OUTPUT_RATE = 5f;

    	// Token: 0x04000A8E RID: 2702
    	private const float TARGET_OUTPUT_TEMPERATURE = 313.15f;
    }

Create a new csharp class file in VS and copy the content into it.
Remember to rename

  • the class name
  • id in method CreateBuildingDef()
  • ID in class
    In this example change them to GoldWasher
  1. Add GoldWasher to build menu.
    create a new patch for GeneratedBuildings
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    using System;
    using HarmonyLib;

    namespace ONI_CuppyUniverse_Mod
    {
        [HarmonyPatch(typeof(GeneratedBuildings), "LoadGeneratedBuildings")]
        public class GeneratedBuildings_LoadGeneratedBuildings
        {
            public static void Prefix()
            {
                Strings.Add(new string[] { 
                    "STRINGS.BUILDINGS.PREFABS.GOLDWASHER.NAME",
                    "淘金器"
                });
                Strings.Add(new string[] {
                    "STRINGS.BUILDINGS.PREFABS.GOLDWASHER.EFFECT",
                    "从纯净水中淘取黄金"
                });
                Strings.Add(new string[] {
                    "STRINGS.BUILDINGS.PREFABS.GOLDWASHER.DESC",
                    "你是信无中生有的"
                });
                ModUtil.AddBuildingToPlanScreen("Food", "GoldWasher");
            }
        }
    }

Reference

m1
Posted Updated 5 minutes read (About 766 words)

m1

Task

  • Load car configuration from .json file
  • Compose DevDocs
  • Investigate csharp comment documentation
  • use assimp to load .glb model

Vehicle Physics Pro

Starting point
The unity’s build-in physics engine PhysX is far from being precise enough for a vehicle simulator.

Needs

  • Closer to reality vehicle physics
  • Runtime performance
  • Not so complicated to config so that the composition of the savings and editor will not be a headache

Official

Install

Through Asset Store

GLTFUtility

Starting point
The car model should be customizable, i.e. the user drag a car model into a directory and the game should be able to load it.
Filetype .glb is nice but unity doesn’t have build-in support

Needs

  • Runtime import
  • support mesh and material

GLTFUtility

Install

In tactics project:

1
git submodule add https://github.com/Siccity/GLTFUtility.git ./Tactics/Assets/ThirdParty/GLTFUtility

Runtime API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
using Siccity.GLTFUtility

// Single thread
void ImportGLTF(string filepath){
    GameObject res = Importer.LoadFromFile(filepath);
}

// Multi thread
void ImportGLTFAsync(string filepath){
    Importer.ImportGLTFAsync(filepath, new ImportSettings(), OnFinish);
}
void OnFinish(){
    ...
}

here I use Application.streamingAssetsPath+"/Model/"+JsonReader.vehicle.model.carBody[0].dir as filepath

Trouble shooting

  1. In the built game, encounter ArgumentNullException: Value cannot be null. Parameter name: shader
    Github Issue
  2. The file management of the game changed after being built. How to determine where to place the Save and Model folders?
    Use Application.streamingAssetsPath, stands for Assets/StreamingAssets in Unity editor and Game_Data/StreamingAssets in exported game.

Csharp’s interpretation of Json

First construct the structure of the json file in c#.
Notice: the name of the member variable should be the same with those in json file 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
[System.Serializable]
public class VehiclePhysicsPara
{
    public float bodyMass;
    public Vector3 centerOfMass;
    public List<BodyColliderPara> collider;
    public List<WheelColliderPara> wheel;
}

[System.Serializable]
public class BodyColliderPara
{
    public string type; // box | sphere
    public Vector3 position;
    public Vector3 eulerRotation;
    public Vector3 scale;
}
[System.Serializable]
public class SteeringWheelTypePara
{
    public bool use;
    public bool inverse;
}
[System.Serializable]
public class WheelTypePara
{
    public bool powered;
    public SteeringWheelTypePara steering;
}
[System.Serializable]
public class WheelSuspensionPara
{
    public float spring;
    public float damper;
    public float distance;
    public float initialPosition;
}
[System.Serializable]
public class WheelFrictionPara
{
    public float extremumSlip;
    public float extremumValue;
    public float AsymptoteSlip;
    public float AsymptoteValue;
}

[System.Serializable]
public class WheelColliderPara
{
    public WheelTypePara type;
    public float mass;
    public float radius;
    public Vector3 position;
    public WheelSuspensionPara suspension;
    public WheelFrictionPara forwardFriction;
    public WheelFrictionPara sidewayFriction;
}

[System.Serializable]
public class VehicleModelPara
{
    public List<CarBodyPara> carBody;
    public List<WheelModelPara> wheel;
}
[System.Serializable]
public class CarBodyPara
{
    public string name;
    public string dir;
    public Vector3 position;
    public Vector3 eulerRotation;
    public Vector3 scale;
    public string shader;
}
[System.Serializable]
public class WheelModelPara
{
    public string name;
    public string dir;
    public Vector3 scale;
    public string shader;
}
[System.Serializable]
public class VehiclePara
{
    public VehicleModelPara model;
    public VehiclePhysicsPara physics;  
}

Notice: [System.Serializable] is for serialized display in Unity editor.

Then, directly use build-in json loader:

1
2
VehiclePara vehicle;
vehicle = JsonUtility.FromJson<VehiclePara>(textJson.text);

All the parameters will be stored in the instance vehicle .

vehiclePara

Abstract the vehicle as a Json structure

Graph

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
flowchart RL
    Main
    Model --> Main
    Physics --> Main
    Controller --> Main
    carBody --> Model
    subgraph bodyPart
    part1
    part2
    end
    bodyPart --> carBody
    wheelVis --> Model
    wheelVisGroup --> wheelVis
    subgraph bodyCollider
        boxCol
        sphereCol
    end
    bodyCollider --> collider
    collider --> Physics
    massProperty --> Physics
    wheelProperty --> Physics
    wheelColGroup --> wheelProperty
    subgraph wheelVisGroup
        tyreVis1
        tyreVis2
        tyreVis3
        tyreVis4
    end
    subgraph wheelColGroup
        wheelCol1 
        wheelCol2 
        wheelCol3 
        wheelCol4 
    end
    wheelVisGroup -- oneToOne --> wheelColGroup

Json example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
{
    "model":{
        "carBody":[
            {
                "name":"body1",
                "dir":"./CustomModel/car1/body1.obj",
                "position":{"x":0,"y":0,"z":0},
                "eulerRotation":{"x":0,"y":0,"z":0},
                "scale":{"x":0,"y":0,"z":0},
                "shader":"Standard"
            },
            {
                "name":"body2",
                "dir":"./CustomModel/car1/body2.obj",
                "position":{"x":0,"y":0,"z":0},
                "eulerRotation":{"x":0,"y":0,"z":0},
                "scale":{"x":0,"y":0,"z":0},
                "shader":"Standard"
            }
        ],
        "wheel":[
            {
                "name":"wheel1",
                "dir":"./CustomModel/car1/wheel1.obj",
                "scale":{"x":0,"y":0,"z":0},
                "shader":"Standard"
            }
        ]
    },
    "physics":{
        "collider":[
            {
                "type":"box",
                "position":{"x":0,"y":0,"z":0},
                "eulerRotation":{"x":0,"y":0,"z":0},
                "scale":{"x":0,"y":0,"z":0}
            },
            {
                "type":"sphere",
                "position":{"x":0,"y":0,"z":0},
                "eulerRotation":{"x":0,"y":0,"z":0},
                "scale":{"x":0,"y":0,"z":0}
            }
        ],
        "centerOfMass":{"x":0,"y":0,"z":0},
        "bodyMass":700,
        "wheel":[
            {
                "type":{"powered":true,"steering":{"use":true,"inverse":false}},
                "mass":7,
                "radius":0.5,
                "position":{"x":0,"y":0,"z":0},
                "suspension":{
                    "spring":25000,
                    "damper":9000,
                    "distance":0.3,
                    "initialPosition":0.5
                },
                "forwardFriction":{
                    "extremumSlip":0.1,
                    "extremumValue:":1,
                    "AsymptoteSlip":0.2,
                    "AsymptoteValue":0.8
                },
                "sidewayFriction":{
                    "extremumSlip":0.05,
                    "extremumValue:":1,
                    "AsymptoteSlip":0.2,
                    "AsymptoteValue":0.8
                }
            }
        ]
    }
}
Follow

Archives

Recents

Tags

6-D3
AI9
AIGC1
AR1
Behaviorism1
CADC1
CD1
CNN1
CSharp1
CV8
Capstone10
Control1
DD1
DT1
Discrete-Mathematics1
ESP322
Education1
Embeded-System9
FL1
Federated-Learning1
Foundation1
Functional programming1
GPT3
Game5
Gated-NN2
Github1
HRI2
HTML1
Haskell1
Hexo2
Humanism1
IK1
ImitationLearning1
Javascript1
LLM1
LSP2
Latex1
LinearAlgebra1
MBTI1
ML5
MR/AR3
Method1
Mod3
Music3
NLP4
NN1
Network2
OpenCV1
PHD1
PSY4
Planning1
Plugin1
PoseEstimation1
Postgraduate1
Prefab1
Probability1
Python1
Pytorch1
QML1
RNN4
ROS3
Reading16
Real2Sim1
Reconstruct3
Robot2
Robotics5
SJTU-Lecture1
SQL2
Script2
Shader3
Signals and Systems1
Sim2Real1
Snippets1
Star-rail1
TC1
TOEFL1
Tech Communication1
Transoformer1
Unity20
VideoEditing1
Vim1
WSL1
Web1
academic-ability1
aliyun1
app3
atlas1
autonomousDriving2
beautify1
bs41
c1
comment&doc1
course1
csharp7
css1
cuda1
dataframe1
debate5
debugger1
docs1
dynamic-programmming1
emoation1
emotion7
ethic1
family1
git5
godot3
health1
hexo7
html4
hyprland2
idea1
javascript3
jolt1
json1
linux14
live2d1
llm1
love1
lua1
mason1
math1
meeting1
meme1
motivation1
movie1
multi-view1
music2
nodejs1
numpy1
nvim8
operational-ability1
oral1
pandas2
path1
philosophy1
physX1
physics-engine1
pio2
plugin7
python17
quantum1
recursive1
regex2
reinforcement-learning1
repository2
scrap1
segmentation2
shell4
sklearn1
society1
ssh2
start-business1
submodule1
supervised-learning2
tasks1
thought1
torch1
travel2
unsupervised-learning1
version-management1
waybar1
wayland1
website1
well-being1
window-manager2
ykll3
zen1
🐱1
×