https://dl.acm.org/doi/pdf/10.1145/3620666.3651367

Background

Qumode

Qumodes are a different way of carrying and manipulating quantum information than qubits.

就像二进制在电脑中的encoding方式，总共n bit的内存，那必然只能有 $2^n$ 种内存state，这是由二进制0或1的特性决定的。然后在让我们看qubit和qumode

States for qubits

If we go to qubits, not much in this picture changes. While a qubit has infinitely many possible states, it turns out that you should look at what is called the basis of the state space, which loosely said means that you should find the minimal number of states in which you can express every other state. For a qubit, this turns out to be two, for example the up state and the down state. To use the language from above, each qubit therefore has 2 ‘possible assignments’, and you have n of them, so by the arguments presented above, there are $2^ⁿ$ unique states. Because we are doing quantum mechanics, superpositions of these states are also allowed, but that doesn’t change the picture: the dimensionality of the system is still $2^ⁿ$.

Qubits是由单个光子的量子态决定的，的存储维数限制依然是 $2^n$

States for qumodes

Squeezing gates

Materials

compilations

Dataset

H3.6m

Human3.6M

Parse

Paper

Depth camera

Doc

Human motion prediction using recurrent neural networks

Repository
Paper

Input

raw data : for each subjects(S1,S2 …) , each action(walking, waiting, smoking …), each sub sequence(1/2):
$(n) \times 99$ (np.ndarray, float32)

From data_utils.load_data() used by translate.read_all_data()

train data: the composed dictionary ((suject_id, action, subaction_id, ‘even’) as key) of raw data (just even rows), with one hot encoding columns for action type, if action is specified (normal case), just append an all 1 column to rawdata. Size of each dictionary value:
$(n/2) \times (99 + actions;count)$

complete data: all data joint together, from different subjects, actions, sub sequences:
$(n) \times 99$

From translate.read_all_data() used by translate.train()

train set : normalized train data, throw out data with $std < 1e-4$ (accroding to complete data). Size of each dictionary value:
$(n/2) \times ((99-used;dimension;count) + actions;count)$

Human Dimension

After the analyzztion of the complete data, human dimension has been fixed to $54$.

From Seq2SeqModel.get_batch() used by translate.train()

total_seq: $60$ ($[0,59]$)

source_seq_len: $50$
target_seq_len: $10$

batch_size: $16$

encoder_inputs: $16\times 49\times (54+actions;count)$
Interpretation: [batch,frame,dimension]
frame range: $[0,48]$

decoder_inputs: $16\times 10\times (54+actions;count)$
frame range: $[49,58]$

decoder_outputs: $16\times 10\times (54+actions;count)$
frame range: $[50,59]$

Model prediction input

encoder_inputs: Tensor form of encoder_inputs from Seq2SeqModel.get_batch()

1

torch.from_numpy(encoder_inputs).float()

decoder_inputs: Tensor form of decoder_inputs from Seq2SeqModel.get_batch()

Example

For detailed usage, please see [Adopted] human-motion-prediction-pytorch\src\predict.ipynb

Reminder

The kinect camera’s output is not guaranteed to be consistent with the input of this model (some features are cut off), so further research is needed.

Pykinect

Run pyKinectAzure\examples\exampleBodyTrackingTransformationComparison to get the camera output record in pyKinectAzure\saved_data, saved as .npy