• Environment
• Example

Environment

Install package for sql

1
2
3

pip install sqlalchemy
pip install ibm_db_sa
pip install ipython-sql

Example

• Python related
  • Magma
  • Jupynium

Python related

Magma

magma-nvim
Magma is a NeoVim plugin for running code interactively with Jupyter.

Jupynium

Jupynium.nvim
It’s just like a markdown live preview, but it’s Jupyter Notebook live preview!

Jupynium uses Selenium to automate Jupyter Notebook, synchronising everything you type on Neovim.
Never leave Neovim. Switch tabs on the browser as you switch files on Neovim.

Catalog

• Introduce
  • Motivation
  • Objective
• Implementation
  • Environment Preparation
  • Foundation
  • Code Review
    • Function detect_target
      • Overview
      • Get the estimated area of the target by color filtering

Introduce

Motivation

We want to apply computer vision to the plane to fullfill ground investigation task automatically.

Objective

The input image looks like this:

And we hope the output prediction to be 56.

Implementation

Environment Preparation

• python3
• numpy
• torch
• torchvision
• opencv-python
• other basic or dependency package

(requirment.txt will be added afterwards)

Foundation

Basic OpenCV methods is required.

Recommended learning site:

• For OpenCV
• • Bilibili video tutorial (Only 1-27 is required for this project)
• • Offical document
• For CNN(implemented by pytorch)
• • What is CNN
• • Official tutorial
• • Hand -written number recognition

Code Review

Source code
in detect.py, there are two large function

1
2
3
4

def detect_target(image):
    ...
def get_numberShot(img, rect_list, ROI):
    ...

Function detect_target

Overview

detect_target receive image data(three channels, BGR), detect the colored targets and returns the index of them.

index:

• rect_list
• • the list whose elements are Box2D type rect, indicating the information of the minimum bounding rectangle
• ROI
• • the list whose elements are index([x,y,w,h]) of orthogonal bounding rectangle
    
    In this picture, the blue frames are the minimum bounding Rectangle and the green ones are ordinary bounding box.

Note: What is Box2D type rect?

Get the estimated area of the target by color filtering

First we need to convert the color layout from BGR to HSV, which is easier to judge color.

1

image_hsv=cv2.cvtColor(image_cut,cv2.COLOR_BGR2HSV)

Then define a mask that filter the color(hue) range from 160-179 or 0-10 (red),

1
2
3
4
5
6
7

red1=np.array([0,100,100])
red2=np.array([10,255,255])
red3=np.array([160,100,100])
red4=np.array([179,255,255])
mask1=cv2.inRange(image_hsv,red1,red2)
mask2=cv2.inRange(image_hsv,red3,red4)
mask=cv2.bitwise_or(mask1,mask2)

And apply the mask to the picture

1

after_mask=cv2.add(image_cut, np.zeros(np.shape(image_cut), dtype=np.uint8), mask=mask)

Turn the after mask image into binary image and adapt close and open processing to fill holes and cancel noise.

1
2
3

kernel=np.ones([3,3],np.uint8)
close=cv2.morphologyEx(binary,cv2.MORPH_CLOSE,kernel,iterations=3)
Open=cv2.morphologyEx(close,cv2.MORPH_OPEN,kernel,iterations=1)

Then, find the contours of the processed image.

For each contour, adapt filters such as area, the ratio of the height and width of bounding rectangle.

If these restrictions are fullfilled, append the index of bounding rectangle and min area rectangle to two lists respectively, and return the lists.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

contours, hier=cv2.findContours(Open,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE)
res_img=image.copy()
ROI=[]
rect_list=[]
for i in range(len(contours)):
    if cv2.contourArea(contours[i])<100:
        continue
    epsl=0.01*cv2.arcLength(contours[i],True)
    approx=cv2.approxPolyDP(contours[i],epsl,True)
    x,y,w,h=cv2.boundingRect(approx)
    if (float(w/h)<2) and (float(w/h)>0.5):
        res_img=cv2.rectangle(res_img,(x,y),(x+w,y+h),(0,255,0),1)
        ROI.append([x,y,w,h])
        rect = cv2.minAreaRect(approx)
        box = cv2.boxPoints(rect)
        box = np.int0(box)
        rect_list.append(rect)
        res_img=cv2.drawContours(res_img, [box], 0, (255, 0, 0), 2)
cv2.imshow('res_img',res_img)
return rect_list,ROI