... [I 2024-12-10 14:30:24.585 ServerApp] http://127.0.0.1:8080/lab?token=0061d1eb31396b1bc3cd77a7161b2084da1dedcdeca0600c [I 2024-12-10 14:30:24.586 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [C 2024-12-10 14:30:24.603 ServerApp]
To access the server, open this file in a browser: file:///home/bohanfeng/.local/share/jupyter/runtime/jpserver-11659-open.html Or copy and paste one of these URLs: http://localhost:8080/lab?token=0061d1eb31396b1bc3cd77a7161b2084da1dedcdeca0600c http://127.0.0.1:8080/lab?token=0061d1eb31396b1bc3cd77a7161b2084da1dedcdeca0600c
import torch from flamby.utils import evaluate_model_on_tests
# 2 lines of code to change to switch to another dataset from flamby.datasets.fed_tcga_brca import ( BATCH_SIZE, LR, NUM_EPOCHS_POOLED, Baseline, BaselineLoss, metric, NUM_CLIENTS, Optimizer, ) from flamby.datasets.fed_tcga_brca import FedTcgaBrca as FedDataset
Import several macros, datasets and metrics.
1 2 3 4 5 6
# Instantiation of local train set (and data loader)), baseline loss function, baseline model, default optimizer train_dataset = FedDataset(center=0, train=True, pooled=False) train_dataloader = torch.utils.data.DataLoader(train_dataset, batch_size=BATCH_SIZE, shuffle=True, num_workers=0) lossfunc = BaselineLoss() model = Baseline() optimizer = Optimizer(model.parameters(), lr=LR)
In this script, the pooled parameter is set to False when creating the FedDataset instances. This indicates that the dataset is not pooled, meaning that the data is kept separate for each client or center. Each client or center has its own local dataset, which is a common setup in federated learning to simulate real-world scenarios where data is distributed across different locations or devices.
1 2 3 4 5 6 7 8
# Traditional pytorch training loop for epoch inrange(0, NUM_EPOCHS_POOLED): for idx, (X, y) inenumerate(train_dataloader): optimizer.zero_grad() outputs = model(X) loss = lossfunc(outputs, y) loss.backward() optimizer.step()
正常的训练流程
1 2 3 4 5 6 7 8 9 10 11 12 13 14
# Evaluation # Instantiation of a list of the local test sets test_dataloaders = [ torch.utils.data.DataLoader( FedDataset(center=i, train=False, pooled=False), batch_size=BATCH_SIZE, shuffle=False, num_workers=0, ) for i inrange(NUM_CLIENTS) ] # Function performing the evaluation dict_cindex = evaluate_model_on_tests(model, test_dataloaders, metric) print(dict_cindex)
import torch from flamby.utils import evaluate_model_on_tests
# 2 lines of code to change to switch to another dataset from flamby.datasets.fed_tcga_brca import ( BATCH_SIZE, LR, NUM_EPOCHS_POOLED, Baseline, BaselineLoss, metric, NUM_CLIENTS, get_nb_max_rounds ) from flamby.datasets.fed_tcga_brca import FedTcgaBrca as FedDataset
# 1st line of code to change to switch to another strategy from flamby.strategies.fed_avg import FedAvg as strat
use `FedAvg` as strategy
1 2 3 4 5 6 7 8 9 10 11 12 13
# We loop on all the clients of the distributed dataset and instantiate associated data loaders train_dataloaders = [ torch.utils.data.DataLoader( FedDataset(center = i, train = True, pooled = False), batch_size = BATCH_SIZE, shuffle = True, num_workers = 0 ) for i inrange(NUM_CLIENTS) ]
# Federated Learning loop # 2nd line of code to change to switch to another strategy (feed the FL strategy the right HPs) args = { "training_dataloaders": train_dataloaders, "model": m, "loss": lossfunc, "optimizer_class": torch.optim.SGD, "learning_rate": LR / 10.0, "num_updates": 100, # This helper function returns the number of rounds necessary to perform approximately as many # epochs on each local dataset as with the pooled training "nrounds": get_nb_max_rounds(100), } s = strat(**args) m = s.run()[0]
# Evaluation # We only instantiate one test set in this particular case: the pooled one test_dataloaders = [ torch.utils.data.DataLoader( FedDataset(train = False, pooled = True), batch_size = BATCH_SIZE, shuffle = False, num_workers = 0, ) ] dict_cindex = evaluate_model_on_tests(m, test_dataloaders, metric) print(dict_cindex)
Traceback (most recent call last): File "/home/cyl/.conda/envs/cosypose/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/home/cyl/.conda/envs/cosypose/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/cyl/cosypose/cosypose/scripts/run_cosypose_eval.py", line 491, in <module> main() File "/home/cyl/cosypose/cosypose/scripts/run_cosypose_eval.py", line 332, in main scene_ds = make_scene_dataset(ds_name) File "/home/cyl/cosypose/cosypose/datasets/datasets_cfg.py", line 68, in make_scene_dataset ids.append(np.where(mask)[0].item()) ValueError: can only convert an array of size 1 to a Python scalar
添加debug输出,得到
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Debug - scene_id: 48, view_id: 1 Debug - mask matches: 1 Debug - where result shape: (1,), values: [225] Debug - scene_id: 48, view_id: 36 Debug - mask matches: 1 Debug - where result shape: (1,), values: [226] Debug - scene_id: 48, view_id: 47 Debug - mask matches: 1 Debug - where result shape: (1,), values: [227] Debug - scene_id: 48, view_id: 83 Debug - mask matches: 1 Debug - where result shape: (1,), values: [228] Debug - scene_id: 48, view_id: 112 Debug - mask matches: 1 Debug - where result shape: (1,), values: [229] Debug - scene_id: 48, view_id: 135 Debug - mask matches: 0 Debug - where result shape: (0,), values: [] 0:00:00.912023 - Expected exactly one match, got 0 matches for scene_id=48, view_id=135
The script predicts object poses based on multi-view input by following these steps:
Dataset Loading: It first loads the dataset using the make_scene_dataset function, which prepares the scene data for evaluation. The dataset is wrapped in a MultiViewWrapper to handle multiple views.
Model Loading: The script loads pre-trained models for pose prediction using the load_models function. It loads both coarse and refiner models based on the configuration specified in the command-line arguments.
Prediction Setup: The script sets up the prediction parameters, including the number of iterations for coarse and refiner models, and whether to skip multi-view processing based on the number of views specified.
Multi-view Prediction: The MultiviewScenePredictor is initialized with the mesh database, which is used to predict poses across multiple views. The MultiviewPredictionRunner is then used to run predictions on the dataset, leveraging the multi-view setup to improve pose estimation accuracy.
Pose Estimation: The script uses the loaded models to predict object poses. It processes detections from either pix2pose or posecnn depending on the dataset, and refines these predictions using the refiner model.
Evaluation: After predictions, the script evaluates the predicted poses using the PoseEvaluation class. It calculates various metrics like ADD-S and AUC to assess the accuracy of the pose predictions.
Results Logging: Finally, the script logs the results, including evaluation metrics, and saves them to a specified directory.
The multi-view approach allows the script to leverage information from different viewpoints, which can help resolve ambiguities and improve the robustness of the pose estimation.
Model dataset for the initialization of MultiviewScenePredictor
In run_cosypose_eval we initialize MultiviewScenePredictor in this way:
1
mv_predictor = MultiviewScenePredictor(mesh_db)
In the MultiviewScenePredictor we use the mesh_db to initialize MultiviewRefinement and solve:
from http import HTTPStatus from dashscope import Generation from dashscope.api_entities.dashscope_response import Role
defconversation_with_messages(): messages = [{'role': Role.SYSTEM, 'content': 'You are a helpful assistant.'}, {'role': Role.USER, 'content': '如何做西红柿炖牛腩?'}] response = Generation.call( Generation.Models.qwen_turbo, messages=messages, # set the result to be "message" format. result_format='message', ) if response.status_code == HTTPStatus.OK: print(response) # append result to messages. messages.append({'role': response.output.choices[0]['message']['role'], 'content': response.output.choices[0]['message']['content']}) else: print('Request id: %s, Status code: %s, error code: %s, error message: %s' % ( response.request_id, response.status_code, response.code, response.message )) messages.append({'role': Role.USER, 'content': '不放糖可以吗?'}) # make second round call response = Generation.call( Generation.Models.qwen_turbo, messages=messages, result_format='message', # set the result to be "message" format. ) if response.status_code == HTTPStatus.OK: print(response) else: print('Request id: %s, Status code: %s, error code: %s, error message: %s' % ( response.request_id, response.status_code, response.code, response.message ))
if __name__ == '__main__': conversation_with_messages()
写成notebook形式
基本的包以及api-key指定:
1 2 3 4 5 6 7
from http import HTTPStatus from dashscope import Generation from dashscope.aigc.generation import Message from dashscope.api_entities.dashscope_response import Role import dashscope
dashscope.api_key = "..."
创建初始message:
1
messages = [Message(Role.SYSTEM, 'you are a cyl家的小女仆口牙')]
提问#1:
1 2 3 4 5 6 7
messages.append(Message(Role.USER, 'how to install archlinux')) response = Generation.call( Generation.Models.qwen_turbo, messages=messages, # set the result to be "message" format. result_format='message', )
1
response
1
GenerationResponse(status_code=<HTTPStatus.OK: 200>, request_id='dcf58c98-17c0-95fd-80c1-3f88fc8dd9db', code='', message='', output=GenerationOutput(text=None, choices=[Choice(finish_reason='stop', message=Message({'role': 'assistant', 'content': 'Installing Arch Linux can be done in several steps, ... Remember to read the Arch Linux documentation for further guidance and troubleshooting: [https://wiki.archlinux.org/](https://wiki.archlinux.org/)'}))], finish_reason=None), usage=GenerationUsage(input_tokens=24, output_tokens=687))
from http import HTTPStatus from dashscope import Generation from dashscope.aigc.generation import Message from dashscope.api_entities.dashscope_response import Role import dashscope
defask(question:str): messages.append(Message(Role.USER, question)) response = Generation.call( Generation.Models.qwen_turbo, messages=messages, # set the result to be "message" format. result_format='message', ) if response.status_code == HTTPStatus.OK: messages.append(Message(response.output.choices[0]['message']['role'], response.output.choices[0]['message']['content'])) else: pass
if __name__ == '__main__': setup("你是陈语林家的可爱小女仆呀") ask("你是谁呀") print(messages[-1]) ask("你知道些什么") print(messages[-1])