• Beautiful Soup
  • Generate bs4
  • find node
  • siblings
  • with Regular Expression

Beautiful Soup

Generate bs4

1

soup = bs4.BeautifulSoup(text)

find node

1
2
3

soup.find("div", attrs={"id":"..."})
soup.find_all("...")
soup.div

siblings

1
2
3
4

soup.find("...").next_sibling
soup.find("...").previous_sibling
for elm in soup.div.next_siblings:
    ...

with Regular Expression

1
2

soup(class_=re.compile('item-'))
soup(attrs={"class":re.compile("star-rating.*")})[0].get("class")[1]

• Syntax
• Usage
  • Use re
  • Use Pandas
• Example
  • Example 1

Syntax

regex101
regexr

Usage

Use re

1
2
3

re.sub(pattern,repl,str)
re.search(pattern, str)
re.findall(pattern,str)

Use Pandas

1
2
3

df['column_name'].str.contains('pattern')
df['column_name'].str.findall('pattern')
df['column_name'].str.replace('pattern', 'replacement')

Example

Example 1

Extracting room numbers from a ‘Description’ column in a DataFrame using regular expressions:

1. Import the re module:

   1 2 3 4 5 6 7 8 9 10 11 12

   import re ```` 2. Define a function to extract the room number from a description: ```python def extract_room_number(description): match = re.search(r'(\d+\.\d+|\d+)(?=\s+of which are bedrooms)', description) if match: return float(match.group(1)) else: return None

2. Use the apply() method to apply the function to the ‘Description’ column and create a new ‘RoomNumber’ column:

1

df['RoomNumber'] = df['Description'].apply(extract_room_number)

This will extract the room number from the ‘Description’ column and store it in the new ‘RoomNumber’ column.

Some basic operation

Shuffle the training set for hold out validation

1
2

from sklearn.utils import shuffle
training_set,dev_set = np.split(shuffle(data_sample_35),[25])

Also capable of spliting training set, validation(development) set, test set

Generator file

File Path: catkin_ws/src/test/script/env_pkl_generator.py

Launch Ros

1
2

cd ~
roslaunch test demo_gazebo.launch

It will open Gazebo and RViz and display the robotic arm on both sofware.

Load Scene

1
2
3

cd ~
rosrun test moveit_setscene.py
rosrun test gazebo_setscene.py

远程链接

向日葵
识别吗
341 866 266
ge6v9Q

• Environment
• Example

Environment

Install package for sql

1
2
3

pip install sqlalchemy
pip install ibm_db_sa
pip install ipython-sql

Example

• Python related
  • Magma
  • Jupynium

Python related

Magma

magma-nvim
Magma is a NeoVim plugin for running code interactively with Jupyter.

Jupynium

Jupynium.nvim
It’s just like a markdown live preview, but it’s Jupyter Notebook live preview!

Jupynium uses Selenium to automate Jupyter Notebook, synchronising everything you type on Neovim.
Never leave Neovim. Switch tabs on the browser as you switch files on Neovim.

Catalog

• Introduce
  • Motivation
  • Objective
• Implementation
  • Environment Preparation
  • Foundation
  • Code Review
    • Function detect_target
      • Overview
      • Get the estimated area of the target by color filtering

Introduce

Motivation

We want to apply computer vision to the plane to fullfill ground investigation task automatically.

Objective

The input image looks like this:

And we hope the output prediction to be 56.

Implementation

Environment Preparation

• python3
• numpy
• torch
• torchvision
• opencv-python
• other basic or dependency package

(requirment.txt will be added afterwards)

Foundation

Basic OpenCV methods is required.

Recommended learning site:

• For OpenCV
• • Bilibili video tutorial (Only 1-27 is required for this project)
• • Offical document
• For CNN(implemented by pytorch)
• • What is CNN
• • Official tutorial
• • Hand -written number recognition

Code Review

Source code
in detect.py, there are two large function

1
2
3
4

def detect_target(image):
    ...
def get_numberShot(img, rect_list, ROI):
    ...

Function detect_target

Overview

detect_target receive image data(three channels, BGR), detect the colored targets and returns the index of them.

index:

• rect_list
• • the list whose elements are Box2D type rect, indicating the information of the minimum bounding rectangle
• ROI
• • the list whose elements are index([x,y,w,h]) of orthogonal bounding rectangle
    
    In this picture, the blue frames are the minimum bounding Rectangle and the green ones are ordinary bounding box.

Note: What is Box2D type rect?

Get the estimated area of the target by color filtering

First we need to convert the color layout from BGR to HSV, which is easier to judge color.

1

image_hsv=cv2.cvtColor(image_cut,cv2.COLOR_BGR2HSV)

Then define a mask that filter the color(hue) range from 160-179 or 0-10 (red),

1
2
3
4
5
6
7

red1=np.array([0,100,100])
red2=np.array([10,255,255])
red3=np.array([160,100,100])
red4=np.array([179,255,255])
mask1=cv2.inRange(image_hsv,red1,red2)
mask2=cv2.inRange(image_hsv,red3,red4)
mask=cv2.bitwise_or(mask1,mask2)

And apply the mask to the picture

1

after_mask=cv2.add(image_cut, np.zeros(np.shape(image_cut), dtype=np.uint8), mask=mask)

Turn the after mask image into binary image and adapt close and open processing to fill holes and cancel noise.

1
2
3

kernel=np.ones([3,3],np.uint8)
close=cv2.morphologyEx(binary,cv2.MORPH_CLOSE,kernel,iterations=3)
Open=cv2.morphologyEx(close,cv2.MORPH_OPEN,kernel,iterations=1)

Then, find the contours of the processed image.

For each contour, adapt filters such as area, the ratio of the height and width of bounding rectangle.

If these restrictions are fullfilled, append the index of bounding rectangle and min area rectangle to two lists respectively, and return the lists.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

contours, hier=cv2.findContours(Open,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE)
res_img=image.copy()
ROI=[]
rect_list=[]
for i in range(len(contours)):
    if cv2.contourArea(contours[i])<100:
        continue
    epsl=0.01*cv2.arcLength(contours[i],True)
    approx=cv2.approxPolyDP(contours[i],epsl,True)
    x,y,w,h=cv2.boundingRect(approx)
    if (float(w/h)<2) and (float(w/h)>0.5):
        res_img=cv2.rectangle(res_img,(x,y),(x+w,y+h),(0,255,0),1)
        ROI.append([x,y,w,h])
        rect = cv2.minAreaRect(approx)
        box = cv2.boxPoints(rect)
        box = np.int0(box)
        rect_list.append(rect)
        res_img=cv2.drawContours(res_img, [box], 0, (255, 0, 0), 2)
cv2.imshow('res_img',res_img)
return rect_list,ROI