• Creating a DataFrame
• Location
• type transforming
• Groupby and agg
• Example
  • Example 1

Creating a DataFrame

1
2
3
4

import pandas as pd
df = pd.read_csv("a.csv")
df = pd.DataFrame({dict_t}) #convert a dict to a dataframe
df = Series_t.to_frame()

Location

1
2
3
4
5
6
7

df.loc[0:4,"Year":"Party"]
df.loc[[1,2,5],["Year","Candidate"]]
df.loc[:,["Year","Candidate"]]
df.loc[:,"year"] # return series
df.loc[1,"year":"Party"] # return series
df.shape[0] # count of row
df[["Year","Candidate"]][0:5]

type transforming

1
2
3

df['column_name'] = df['column_name'].astype(int)
df['column_name'] = df['column_name'].astype(float)
df['column_name'] = df['column_name'].astype(str)

Groupby and agg

1
2
3

df.groupby('column_name').agg(['sum', 'mean', 'max', 'min'])
df.groupby(['column_name_1', 'column_name_2']).agg({'column_name_3': ['sum', 'mean'], 'column_name_4': ['max', 'min']})
df.groupby(['column_name_1', 'column_name_2']).agg(['sum', 'mean', 'max', 'min'])

Example

Example 1

Extracting the top 20 categories from a DataFrame using groupby:

1. Use the groupby and size methods to calculate the size of each group:

   1 2 3 4 5 6 7

   group_sizes = df.groupby('category').size() ```` 2. Use the `sort_values` method to sort the group sizes in descending order: ```python sorted_groups = group_sizes.sort_values(ascending=False)

2. Use the head method to extract the top 20 categories:

1

top_20 = sorted_groups.head(20)

4. Use the isin method to filter the original DataFrame to only include rows with the top 20 categories:

1

df_filtered = df[df['category'].isin(top_20.index)]

This will extract the top 20 categories from the ‘category’ column of the DataFrame and create a new DataFrame that only includes rows with those categories.

Summer Camp

Announcements

笔试科目说明

公告

• Beautiful Soup
  • Generate bs4
  • find node
  • siblings
  • with Regular Expression

Beautiful Soup

Generate bs4

1

soup = bs4.BeautifulSoup(text)

find node

1
2
3

soup.find("div", attrs={"id":"..."})
soup.find_all("...")
soup.div

siblings

1
2
3
4

soup.find("...").next_sibling
soup.find("...").previous_sibling
for elm in soup.div.next_siblings:
    ...

with Regular Expression

1
2

soup(class_=re.compile('item-'))
soup(attrs={"class":re.compile("star-rating.*")})[0].get("class")[1]

• Syntax
• Usage
  • Use re
  • Use Pandas
• Example
  • Example 1

Syntax

regex101
regexr

Usage

Use re

1
2
3

re.sub(pattern,repl,str)
re.search(pattern, str)
re.findall(pattern,str)

Use Pandas

1
2
3

df['column_name'].str.contains('pattern')
df['column_name'].str.findall('pattern')
df['column_name'].str.replace('pattern', 'replacement')

Example

Example 1

Extracting room numbers from a ‘Description’ column in a DataFrame using regular expressions:

1. Import the re module:

   1 2 3 4 5 6 7 8 9 10 11 12

   import re ```` 2. Define a function to extract the room number from a description: ```python def extract_room_number(description): match = re.search(r'(\d+\.\d+|\d+)(?=\s+of which are bedrooms)', description) if match: return float(match.group(1)) else: return None

2. Use the apply() method to apply the function to the ‘Description’ column and create a new ‘RoomNumber’ column:

1

df['RoomNumber'] = df['Description'].apply(extract_room_number)

This will extract the room number from the ‘Description’ column and store it in the new ‘RoomNumber’ column.

Some basic operation

Shuffle the training set for hold out validation

1
2

from sklearn.utils import shuffle
training_set,dev_set = np.split(shuffle(data_sample_35),[25])

Also capable of spliting training set, validation(development) set, test set

Generator file

File Path: catkin_ws/src/test/script/env_pkl_generator.py

Launch Ros

1
2

cd ~
roslaunch test demo_gazebo.launch

It will open Gazebo and RViz and display the robotic arm on both sofware.

Load Scene

1
2
3

cd ~
rosrun test moveit_setscene.py
rosrun test gazebo_setscene.py

远程链接

向日葵
识别吗
341 866 266
ge6v9Q

Practicing website

sql-practice

• Environment
• Example

Environment

Install package for sql

1
2
3

pip install sqlalchemy
pip install ibm_db_sa
pip install ipython-sql

Example

• 如何使用Shader实现语义分割
  • 什么是Shader
  • Shader代码的组成部分
  • 如何使用Shader实现语义分割

如何使用Shader实现语义分割

什么是Shader

Shader是一种计算机程序，它用于在图形渲染管线中处理顶点和像素。在Unity中，Shader可以用来控制物体的渲染方式，包括颜色、纹理、透明度等。Shader的工作流程是将材料（如纹理、数据、颜色等）通过加工展现在材质上的过程。

Unity中有多种类型的Shader，可以根据不同的情况使用。例如，在三维网格上使用的方式和图片上使用的方式会有区别。

Shader代码的组成部分

一个Unity Shader通常由以下几个部分组成：

• Properties：这部分定义了Shader中可供用户在材质检查器中编辑的属性。例如，可以定义一个颜色属性，让用户可以在材质检查器中更改物体的颜色。

• SubShader：每个Shader至少包含一个SubShader。SubShader定义了Shader的实际渲染操作。如果Shader需要支持多种不同的渲染硬件或渲染质量设置，可以在Shader中定义多个SubShader。

• Pass：每个SubShader至少包含一个Pass。Pass定义了一次渲染操作，包括顶点和片段着色器的代码。可以在一个SubShader中定义多个Pass来实现多次渲染操作。

• CGPROGRAM：这部分包含了顶点和片段着色器的代码。可以使用Cg/HLSL语言来编写着色器代码，并使用内置的变量和函数来访问物体的信息。

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

  Shader "Custom/SimpleColor" { Properties { _Color ("Color", Color) = (1,1,1,1) } SubShader { Tags { "RenderType"="Opaque" } LOD 100 Pass { CGPROGRAM #pragma vertex vert #pragma fragment frag fixed4 _Color; struct appdata { float4 vertex : POSITION; }; struct v2f { float4 vertex : SV_POSITION; }; v2f vert (appdata v) { v2f o; o.vertex = UnityObjectToClipPos(v.vertex); return o; } fixed4 frag (v2f i) : SV_Target { return _Color; } ENDCG } } }

如何使用Shader实现语义分割

如果希望根据物体的标签来控制物体的颜色，可以在Shader中使用材质属性来实现。首先，需要在脚本中获取物体的标签信息，并根据标签信息设置物体材质的颜色属性。然后，在Shader中，可以使用这个颜色属性来控制物体的渲染颜色。

下面是一个简单的示例，它展示了如何在脚本中设置材质的颜色属性，并在Shader中使用该属性来控制物体的颜色：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

// 脚本中设置材质的颜色属性
void Start()
{
    Renderer renderer = GetComponent<Renderer>();
    Material material = renderer.material;

    // 根据物体的标签设置材质的颜色属性
    if (CompareTag("Pedestrian"))
    {
        material.SetColor("_Color", Color.blue);
    }
    else if (CompareTag("Road"))
    {
        material.SetColor("_Color", Color.white);
    }
    else if (CompareTag("Building"))
    {
        material.SetColor("_Color", Color.black);
    }
}
````

```c
// Shader中使用颜色属性来控制物体的颜色
Shader "Custom/SemanticSegmentation" {
    Properties {
        _Color ("Color", Color) = (1,1,1,1)
    }
    SubShader {
        Tags { "RenderType"="Opaque" }
        LOD 100

        Pass {
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag

            fixed4 _Color;

            struct appdata {
                float4 vertex : POSITION;
            };

            struct v2f {
                float4 vertex : SV_POSITION;
            };

            v2f vert (appdata v) {
                v2f o;
                o.vertex = UnityObjectToClipPos(v.vertex);
                return o;
            }

            fixed4 frag (v2f i) : SV_Target {
                return _Color;
            }
            ENDCG
        }
    }
}

Which are the build-in commands?

A shell built-in command is contained in the shell itself (implemented by shell author) and runs on the same shell without creating a new process, e.g. cd, pwd, exit, alias.

By contrast: For a regular(linux) command, it is run from a binary file (found in $PATH or specified file) by forking the existing process in a subshell, e.g. ls, cat, rm.

How to implement

1. Judge whether is build-in command

Assume we have already got the parsed arguments

char * parsedArgs[MAXPARSE].

For example, a input char * input = "cd .." can be parsed as char * parsedArgs[MAXPARSE] = {"cd", ".."}

Then, we just need to judge whether parsedArgs[0] is contained in our build-in command set {"exit", "cd", "pwd" ...} (by using strcmp)

2. Implementation

In this project (as of milestone 2) , we need to implement exit, pwd, cd.

exit

In Bash, exit is a built-in command used to terminate the current shell session (same in our mumsh).

Each running shell is a process, so we can use the exit() in stdlib.h to terminate the shell process, achieving the goal of exiting the shell.

1
2
3
4

void exec_exit(){
    printf("exit\n");
    exit(0); // 0 for exit normally, otherwise abnormally
}

pwd

Running a crude (or minimal?) shell and have no idea where you are? Just type pwd and it will print the name of the current/working directory.

In C, we can implement the similar functionality using the getcwd() in unistd.h.

if success, getcwd() will return the pointer to the result, else ( e.g. no enough space) return NULL

1
2
3
4

void exec_pwd(){
    char buf[1024]; // memory space need for storage
    printf("%s\n", getcwd(buf,sizeof(buf)));
}

However, the working directory is not guaranteed to be shorter than 1024, so we may need to dynamically allocate the buffer to suit the length.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

int size = 100;
char *buffer = malloc(size);

while (getcwd(buffer, size) == NULL) {
    if (errno == ERANGE) {  // buffer too small
        size *= 2;
        buffer = realloc(buffer, size);
    } else {  // other error
        ...
        free(buffer);
        return 1;
    }
}

printf("%s\n", buffer);

free(buffer);

cd

cd is used for changing working directory.

In C, we can implement the similar functionality using the chdir() in unistd.h.

if success, chdir() will return 0, else ( e.g. no such directory) return -1

1
2
3

void exec_cd(char * path){
    chdir(path);
}

Very straight forward, but if we want to implement more advanced features of the cd command such as changing to the home directory or the previous directory, we would need to handle these cases manually.

For example, you could check if the argument is ~ or -, and then call chdir() with the appropriate path