Materials

compilations

Dataset

H3.6m

Human3.6M

Parse

Paper

Depth camera

Doc

Human motion prediction using recurrent neural networks

Repository
Paper

Input

raw data : for each subjects(S1,S2 …) , each action(walking, waiting, smoking …), each sub sequence(1/2):
$(n) \times 99$ (np.ndarray, float32)

From data_utils.load_data() used by translate.read_all_data()

train data: the composed dictionary ((suject_id, action, subaction_id, ‘even’) as key) of raw data (just even rows), with one hot encoding columns for action type, if action is specified (normal case), just append an all 1 column to rawdata. Size of each dictionary value:
$(n/2) \times (99 + actions;count)$

complete data: all data joint together, from different subjects, actions, sub sequences:
$(n) \times 99$

From translate.read_all_data() used by translate.train()

train set : normalized train data, throw out data with $std < 1e-4$ (accroding to complete data). Size of each dictionary value:
$(n/2) \times ((99-used;dimension;count) + actions;count)$

Human Dimension

After the analyzztion of the complete data, human dimension has been fixed to $54$.

From Seq2SeqModel.get_batch() used by translate.train()

total_seq: $60$ ($[0,59]$)

source_seq_len: $50$
target_seq_len: $10$

batch_size: $16$

encoder_inputs: $16\times 49\times (54+actions;count)$
Interpretation: [batch,frame,dimension]
frame range: $[0,48]$

decoder_inputs: $16\times 10\times (54+actions;count)$
frame range: $[49,58]$

decoder_outputs: $16\times 10\times (54+actions;count)$
frame range: $[50,59]$

Model prediction input

encoder_inputs: Tensor form of encoder_inputs from Seq2SeqModel.get_batch()

1

torch.from_numpy(encoder_inputs).float()

decoder_inputs: Tensor form of decoder_inputs from Seq2SeqModel.get_batch()

Example

For detailed usage, please see [Adopted] human-motion-prediction-pytorch\src\predict.ipynb

Reminder

The kinect camera’s output is not guaranteed to be consistent with the input of this model (some features are cut off), so further research is needed.

Pykinect

Run pyKinectAzure\examples\exampleBodyTrackingTransformationComparison to get the camera output record in pyKinectAzure\saved_data, saved as .npy

• Reference
• Types
  • Supervised learning
  • Unsupervised learning
  • Reinforcement learning
    • Action space
    • Policy
    • Episode
    • Horizon
    • Return
    • Value function
    • Q function

Reference

Deep-Reinforcement-Learning-With-Python

Types

Supervised learning

In supervised learning, the machine learns from training data. The training data consists of a labeled pair of inputs and outputs. So, we train the model (agent) using the training data in such a way that the model can generalize its learning to new unseen data. It is called supervised learning because the training data acts as a supervisor, since it has a labeled pair of inputs and outputs, and it guides the model in learning the given task.

Regression

Quantitative response
predict a quantitative variable from a set of features

Classification

Categorical response
predict a categorical variable

Unsupervised learning

Similar to supervised learning, in unsupervised learning, we train the model (agent) based on the training data. But in the case of unsupervised learning, the training data does not contain any labels; that is, it consists of only inputs and not outputs. The goal of unsupervised learning is to determine hidden patterns in the input. There is a common misconception that RL is a kind of unsupervised learning, but it is not. In unsupervised learning, the model learns the hidden structure, whereas, in RL, the model learns by maximizing the reward.

Reinforcement learning

Action space

The set of all possible actions in the environment is called the action space. Thus, for this grid world environment, the action space will be [up, down, left, right]. We can categorize action spaces into two types:

• Discrete action space
  When our action space consists of actions that are discrete, then it is called a discrete action space. For instance, in the grid world environment, our action space consists of four discrete actions, which are up, down, left, right, and so it is called a discrete action space.
• Continuous action space
  When our action space consists of actions that are continuous, then it is called a continuous action space. For instance, let’s suppose we are training an agent to drive a car, then our action space will consist of several actions that have continuous values, such as the speed at which we need to drive the car, the number of degrees we need to rotate the wheel, and so on. In cases where our action space consists of actions that are continuous, it is called a continuous action space.

Policy

A policy defines the agent’s behavior in an environment. The policy tells the agent what action to perform in each state.
Over a series of iterations, the agent will learn a good policy that gives a positive reward.
The optimal policy tells the agent to perform the correct action in each state so that the agent can receive a good reward.

• Deterministic Policy
  deterministic policy tells the agent to perform a one particular action in a state. Thus, the deterministic policy maps the state to one particular action

• Stochastic Policy
  maps the state to a probability distribution over an action space.

  • Categorical policy
    when the action space is discrete
    uses categorical probability distribution over action space to select actions
  • Gaussian policy
    when our action space is continuous
    the stochastic policy uses Gaussian probability distribution over action space to select actions when the action space is continuous

Episode

The agent interacts with the environment by performing some action starting from the initial state and reach the final state. This agent-environment interaction starting from the initial state until the final state is called an episode. For instance, in the car racing video game, the agent plays the game by starting from the initial state (starting point of the race) and reach the final state (endpoint of the race). This is considered an episode. An episode is also often called trajectory (path taken by the agent)

• Episodic task
  As the name suggests episodic task is the one that has the terminal state. That is, episodic tasks are basically tasks made up of episodes and thus they have a terminal state. Example: Car racing game.
• Continuous task
  Unlike episodic tasks, continuous tasks do not contain any episodes and so they don’t have any terminal state. For example, a personal assistance robot does not have a terminal state.

Horizon

Horizon is the time step until which the agent interacts with the environment. We can classify the horizon into two:

• Finite horizon
  If the agent environment interaction stops at a particular time step then it is called finite Horizon. For instance, in the episodic tasks agent interacts with the environment starting from the initial state at time step t =0 and reach the final state at a time step T. Since the agent environment interaction stops at the time step T, it is considered a finite horizon.
• Infinite horizon
  If the agent environment interaction never stops then it is called an infinite horizon. For instance, we learned that the continuous task does not have any terminal states, so the agent environment interaction will never stop in the continuous task and so it is considered an infinite horizon.

Return

Return is the sum of rewards received by the agent in an episode.

Value function

Value function or the value of the state is the expected return that the agent would get starting from the state $s$ following the policy $\pi$

Q function

implies the expected return agent would obtain starting from the state $s$ and an action $a$ following the policy $\pi$.

Book

Matrix Cook Book

Relationship

Prime theory: An integrated view of motivation and emotion

Motivation

what is my motivation
Intrinsic Motivation and norms about payment

Emotion

what is my emotion
In old days people have high spirits

the obs dictionary’s builder

Report

For me, intrinsic and extrinsic motivation differs a lot. Since my early age, I distinguish clearly between schoolwork and extracurricular activities. During week days, I devote 100% of my energy into schoolwork and during weekends, I participate in robot and model aircraft clubs.
Before I entered college, I think that completing coursework is a student’s duty and it is also very rewarding. By behaving well in school, I can receive praise from my parents, respectation from peers, promising jobs and For this reason, completing schoolwork is mainly motivated by extrinsic rewards.

My dad used to have an argue with my mum about whether they should reward me by giving me some money to buy what I want if I perform well in exams. He thought exam scores should not be related with money rewards. While my mum thought it is reasonable since rewards can motivate me a lot. After I entered college, I thought my dad was right, although money rewards did motivate me to get higher scores, it is extrinsic motivation. According to the textbook, extrinsic rewards can reduce intrinsic motivation. When I look back, I cannot clearly figure out what is my intrinsic motivation of studying hard at school. Most of the time, I am motivated to study a subject just for the reason that I am good at it. If I am not good at a subject, such as Chinese, I just keep persuding me that if my Chinese is not learnt well, I will not be able to enter a good school. Usually, during this process, I suffer from heavy depression and anxiety. Speaking of how much passion I have in all of these subjects, I have to say, very little. My behaviour in exam-oriented education is like a machine. I get rewards, commitments as inputs and output good scores.

Although I behaves like the walking dead in schoolwork, I have intrinsic motivations in other fields, such as building robots and flying model aircraft. These activities fulfill no obvious purpose other than enjoyment.

Big Question

What affects your health?

• Biological, psychological, and social factors all affect your health.
  How does stress affect your health?
• Stress has negative effects on both your physical and mental health.
  How do mediating factors affect your stress?
• Mediating factors can help alleviate the negative effects of stress.
  Can a positive attitude keep you healthy?
• A positive attitude has positive effects on both your physical and mental health.

Biological Factors

Biological factors refer to innate or naturally occurring factors such as genetics, age, and gender that can affect your physical and mental health.

Psychological Factors

Psychological factors refer to factors related to your thoughts and emotions such as emotional state, cognitive processes, and behavioral patterns that can affect your physical and mental health.

Social Factors

Social factors refer to environmental factors such as culture, family, social networks that can affect your physical and mental health.

Stress

Stress refers to tension, anxiety or challenge from internal or external environment which may have negative effects on both physical and mental health.

Mediating Factors

Mediating factors are factors that can help alleviate the negative effects of stress. These factors include positive coping strategies, seeking social support, and changing cognitive processes.

Positive Attitude

A positive attitude has positive effects on both your physical and mental health. It can help you cope with stress, improve your immune system, and increase your overall well-being.

Tips for Maintaining Physical and Mental Health

This section provides tips and suggestions for maintaining physical and mental health. It includes advice on diet, exercise, sleep, relaxation techniques, social networks, and seeking support.

Thoughts

• Health is the constrain

• Learning’s school of thoughts
  • Behaviorism
    • Non-associative
      • Habituation
      • Sensitization
    • Associative
      • Classical conditioning
      • Operant conditioning
    • Watching others
      • Observational learning
      • Modeling
      • Vicarious conditioning
  • Humanism

Reference

• Les Choristes

Learning’s school of thoughts

Take the movie “Les Choristes” as an example

Behaviorism

Principals stand for Behaviorism:

In the face of confused or lost teenagers, the principal’s education method is very representative of the indifferent and rigid “behaviorist”. He believes that violence and punishment can make children keep their duties. The endless and varied punishment methods are his masterpieces. In the face of the principal’s violence, the children really behaved in silence and followed the rules, but they did not believe the principal from the bottom of their hearts. Once the principal leaves and doesn’t pay attention, the children will play tricks behind their backs and vent their emotions.

One of the core viewpoints of behaviorism is that environment is the only condition for the learning, and learning is completely determined by the external environment. Therefore, the principal puts the focus of education on corporal punishment, punishment, reprimand, and criticism. Even if children make small mistakes, they will always be scolded or beaten.

It is true that punishment as an external stimulus can promote children to develop in a good way to a certain extent, but the fear generated by punishment is often greater than the education gained, so children will become more and more disobedient, and they will not learn what they should know at all.

Non-associative

Habituation

Sensitization

Associative

Classical conditioning

When we learn that a stimulus predicts another stimulus.

Operant conditioning

When we learn that a behavior leads to a certain outcome.

Watching others

Observational learning

When we learn or change a behavior after watching a person engage in that behavior

Modeling

Displaying a behavior that imitates a previously observed behavior

Vicarious conditioning

learning about an
action’s consequences by observing others being rewarded or
punished for their behavior

Humanism

Mathews stands for Humanism:
Compared with the principal’s style, Teacher Matthew regards the children as himself, understands their distress, and walks into their hearts. Teacher Matthew is the first to really treat “problem teenagers” as dignified and thoughtful individuals. Teacher Matthew’s education and enlightenment are the classic characteristics of humanists.

Humanism holds that different people will have different beliefs about the same fact, which means different things to different people. Therefore, humanists can always start from the individual’s inner feelings and beliefs, and touch others with their feelings and ideas.

Teacher Matthew pays attention to discovering children’s potential in learning, and constantly encourages children to improve and cheer. Combined with rationality, it only helps children to affirm themselves and realize themselves.

A Message to Myself

hi, how are you doing today?
When you wake up in the morning to the sound of your alarm clock,
when you join an online lecture and hear the hissing sound coming from your headphones,
when the flying catkins take away the smell of heather,
you always wonder why you are here. living.
This is a hard question to answer, you are an atheist, you have to find out the answer that belongs to yourself.
The past 12 months have been very memorable for you, right?
You have changed from an unknown game player to a personal mod developer with over 10,000 subscribers in the steam workshop;
You become an video publisher in bilibili and your first video gets tens of thousands of views;
You have successfully established a blog site of your own;
Based on nvim, you have customized your own fancy editor;
You build a personal knowledge base with obsidian, and you feel your knowledge connected like never before;
You have gained an unforgettable internship experience and got your first salary in life;
You have led many scientific and technological innovation teams and met many people of all kinds;
You got to know chatGPT, which makes you feel that all the knowledge in this world is at your fingertips.

It looks like everything is going well, and you’ve been moving in a positive direction, but it’s superficial, and you still haven’t figured out what you’re living for.
You often boast that you have a good mentality and a big heart, but many times you are not strong psychologically, but numb to respond to stressors.
By now, you have come to the second half of your junior year. Compared with before, the pressure on your shoulders is even greater, especially the pressure of choosing your future path.
Under such pressure, it is easy to get lost if you don’t have clear enough pointers.
So, now is the time to seriously think about what you are seeking for in life. In general, it’s smarter to infer what you’re seeking from what you’re not gonna seek.
First of all, what you are seeking is not a fake, which can be defined as concealment, prevarication, hypocrisy.
For you, personal wealth, schooling certification are all fakes. They can only guide how others judge our worth, but they cannot tell us what our own worth is.

More fundamentally, You don’t pursue the approval of others, and you don’t pursue the society’s expectations of you.
You hate the studies of how to become success, you hate cram school, and you hate these things that are concretized by distorted human nature. You think that people should have some higher purpose.
But those purpose are too high, far beyond your reach, and the devil’s voice of corruption is always in your ears, and your pain comes from it.
In fact, at this point, you already understand what your life goal is, just say it, don’t care what other people will say, at least I won’t laugh at you.
You hope that one day you will contribute, in whatever form, to the advancement of all humanity.
Just like your favorite singer, Michael Jackson, you aspire to be a man who have a world in mind like him.
But now you are too far away from this goal, and you don’t even know how to touch it, but the pressure of life will always follow, and the prejudice of society will not disappear because of your denial.
For example, now, your road to postgraduate education is not clear, what should you do if you encounter various setbacks?
I think what you should do most is to reconcile with yourself. An outstanding person may not necessarily succeed, but he will definitely recognize himself.
Often, you don’t know where the next step will lead you, and you don’t want to put yourself in a bad situation again and again, but you have to remember that I am always by your side,
No matter what you do, no matter how bad things happen, I will always be there for you and I will always approve of you.
As long as the two of us are here, we will find a way to make things better. Because we’re best friends from now on, and that’s what best friends do.

I know a guy who compiled the obsidian version of the Chinese-English thesaurus by himself. He lost his job, got myeloma, and his health and wealth were shattered, but he completed this Grand project under such extremely difficult conditions.
I would like to quote a passage he wrote in his preface:
Now looking back, it all seems like a dream. If I wasn’t poor, I wouldn’t have had the time and energy to do this; if I was rich enough, I might have done something else; if my talent was a little bigger, I wouldn’t have done this; if my talent was a little smaller, it wouldn’t have been enough to do this; the highest level of computer language I know is the most basic HTML language, which can only change the color and style of text; if I hadn’t gotten sick, I wouldn’t have expanded my vocabulary to over nine thousand words; if there was no Bilibili and no video I made myself, my work would not have been known so quickly. All of these seem predetermined, not too much or too little, just right.

When you stumble again and again on the road of life, I will always reconcile with you, and we will face the next challenge with full confidence.

Notice

Acting like my favourite friend.

What do I have (positive aspect)

• I am good at playing sandbox game, maybe a sign of innovative?
• I have learn many skills that can support me to do some individual development
• I have a healthy home environment
• I am not unhealth
• I took part in numoerous project teams and know many people, no matter good or bad
• I have the ability to improve my life through some technical skills

I have fear of what

• 我担心会把实在的当作虚妄的，把虚妄的当成实在的
• 我曾经认可所谓的人本主义教育，作为我对于应试教育的反抗，但现在看来，他也是错的。
• 我担心我一辈子都活在一个乐于对人评头论足的环境
  • 对于一个人的学历
  • 对于一个人的选择
• 我担心我做的事情没有意义
  • 对于我的意义
    • 如果我在人生的最后，回过头来看，我是否会觉得我现在做的毫无意义？
  • 对他人来说的意义
    • 这个已经克服了，但不能完全忽视
    • 比如我的父母，他们在某种程度上就是30年后的我，他们的意见应该参考
• 我担心我对于太多事情有不切实际的期待
  • 对于人的本性（善/恶）
  • 对于人的欲望
  • 对于社会舆论
    • 当我认为他人生活在信息茧房中，我首先需要自指，我自己是否生活在茧房中？
    • 当我认为他人被洗脑时，我是否有被洗脑

Replace fear with confidence

Future is full of possibility

What is my confusion?

• 个人的努力在社会发展的洪流下不值一提

• 关于个人道路的选择

  • 读博士/读硕士
    • 博士？需要大量时间，家庭日渐凋敝，读出来后专业落伍
    • 硕士？内卷，名额太少，学术成就不如博士
    • 我追求学术成就吗？我认为这是在人类史上留下个人价值的最好方式。但我了解到了太多的科研劝退内容。
    • 所谓的劝退，针对的是对于科研抱有不切实际的期望的人群
      • 我属于那个人群吗
  • 学术成就/家财万贯（成功人士）/实业兴邦（打螺丝？）
    • 中产的美梦？
    • 家财万贯不现实，目前社会的上升通道太少，创业失败即家破人亡，ai或许能凿壁偷光？
    • 学术更像是一场赌博，稳妥的办法是添砖加瓦，但这已经有许多人在做了。但许多划时代的研究都不是如此。
    • 我目前的经历更主要偏向应用，调包，调参。。。。
    • 如果是应用，我希望我是构建者而不是码农
    • 什么是构建者？
      • 例如自研一些开发软件

• 我选择的信念是否正确？

  • 我的信念是什么？
    • 在更小时候，我并不喜欢他的音乐风格
    • 在高考那段最磨练意志的时段，主要是MJ的音乐陪我走过，给了我充沛的精神力量
    • 为什么我会喜欢MJ的音乐？抗争的精神力量！
      • 对于肤色的抗争
      • 对于政治迫害的抗争
      • 对于人类欲望的抗争
      • 对于病痛的抗争
    • 那么在高考那段时间，我到底想抗争什么？
      • 极端狭窄的独木桥
      • 对学生的评价体系
      • 压抑的同龄人氛围
      • 将人物化

Summer Camp

Announcements

笔试科目说明

公告

• Creating a DataFrame
• Location
• type transforming
• Groupby and agg
• Example
  • Example 1

Creating a DataFrame

1
2
3
4

import pandas as pd
df = pd.read_csv("a.csv")
df = pd.DataFrame({dict_t}) #convert a dict to a dataframe
df = Series_t.to_frame()

Location

1
2
3
4
5
6
7

df.loc[0:4,"Year":"Party"]
df.loc[[1,2,5],["Year","Candidate"]]
df.loc[:,["Year","Candidate"]]
df.loc[:,"year"] # return series
df.loc[1,"year":"Party"] # return series
df.shape[0] # count of row
df[["Year","Candidate"]][0:5]

type transforming

1
2
3

df['column_name'] = df['column_name'].astype(int)
df['column_name'] = df['column_name'].astype(float)
df['column_name'] = df['column_name'].astype(str)

Groupby and agg

1
2
3

df.groupby('column_name').agg(['sum', 'mean', 'max', 'min'])
df.groupby(['column_name_1', 'column_name_2']).agg({'column_name_3': ['sum', 'mean'], 'column_name_4': ['max', 'min']})
df.groupby(['column_name_1', 'column_name_2']).agg(['sum', 'mean', 'max', 'min'])

Example

Example 1

Extracting the top 20 categories from a DataFrame using groupby:

1. Use the groupby and size methods to calculate the size of each group:

   1 2 3 4 5 6 7

   group_sizes = df.groupby('category').size() ```` 2. Use the `sort_values` method to sort the group sizes in descending order: ```python sorted_groups = group_sizes.sort_values(ascending=False)

2. Use the head method to extract the top 20 categories:

1

top_20 = sorted_groups.head(20)

4. Use the isin method to filter the original DataFrame to only include rows with the top 20 categories:

1

df_filtered = df[df['category'].isin(top_20.index)]

This will extract the top 20 categories from the ‘category’ column of the DataFrame and create a new DataFrame that only includes rows with those categories.

• Beautiful Soup
  • Generate bs4
  • find node
  • siblings
  • with Regular Expression

Beautiful Soup

Generate bs4

1

soup = bs4.BeautifulSoup(text)

find node

1
2
3

soup.find("div", attrs={"id":"..."})
soup.find_all("...")
soup.div

siblings

1
2
3
4

soup.find("...").next_sibling
soup.find("...").previous_sibling
for elm in soup.div.next_siblings:
    ...

with Regular Expression

1
2

soup(class_=re.compile('item-'))
soup(attrs={"class":re.compile("star-rating.*")})[0].get("class")[1]