구차니의 잡동사니 모음

python 안쓰고 c++ 만으로 해보려는데

tensorflow / rosrun 등이 필요함

[링크 : https://github.com/haosen9527/mobileNet-ssd]

빌드는 독립적으로 되는데 caffe를 먼저 빌드는 해야 함..

caffe를 빌드하는데 openCV-3.4.0 이상을 요구하고 있음..

[링크 : https://github.com/MediosZ/MobileNet-SSD]

+

python으로 된 가장 원본 파일?

[링크 : https://github.com/chuanqi305/MobileNet-SSD]

+

tensorflow lite 버전과 openCV-4.1.0 이상 버전 요구?

[링크 : https://github.com/finnickniu/tensorflow_object_detection_tflite]

+

rpi 2b에서 텐서가 되는지 모르겠네?

[링크 : https://www.tensorflow.org/install/source_rpi?hl=ko]

[링크 : https://caffe.berkeleyvision.org/]

외부에서 획득한 cfg와 weight 로 실행을 하려고 하니 엉뚱하게 bicycle,person,car가 나오고 있어서 급 멘붕 -_-

/darknet detect cfg/yolov3.cfg yolov3.weights data/dog.jpg

그래서 코드를 뜯어보니 detect 로 실행 할 경우

test_detector를 실행하게 하고 cfg/coco.data 라는걸 자동으로 사용하게 한다.

    if (0 == strcmp(argv[1], "average")){
        average(argc, argv);
    } else if (0 == strcmp(argv[1], "yolo")){
        run_yolo(argc, argv);
    } else if (0 == strcmp(argv[1], "voxel")){
        run_voxel(argc, argv);
    } else if (0 == strcmp(argv[1], "super")){
        run_super(argc, argv);
    } else if (0 == strcmp(argv[1], "detector")){
        run_detector(argc, argv);
    } else if (0 == strcmp(argv[1], "detect")){
        float thresh = find_float_arg(argc, argv, "-thresh", .24);
		int ext_output = find_arg(argc, argv, "-ext_output");
        char *filename = (argc > 4) ? argv[4]: 0;
        test_detector("cfg/coco.data", argv[2], argv[3], filename, thresh, 0.5, 0, ext_output, 0, NULL, 0, 0);
    }

 coco.data에는 별 건 없는것 같은데 가장 중요한(?)건 names 인 것 같고

$ cat cfg/coco.data
classes= 80
train  = /home/pjreddie/data/coco/trainvalno5k.txt
valid  = coco_testdev
#valid = data/coco_val_5k.list
names = data/coco.names
backup = /home/pjreddie/backup/
eval=coco

coco.names 에는 항목별 이름이 존재하는 것을 확인할 수 있었다.

$ cat data/coco.names
person
bicycle
car
motorbike
aeroplane
bus
train
truck
boat
traffic light
fire hydrant
stop sign
parking meter
bench
bird
cat
dog
horse
sheep
cow
elephant
bear
zebra
giraffe
backpack
umbrella
handbag
tie
suitcase
frisbee
skis
snowboard
sports ball
kite
baseball bat
baseball glove
skateboard
surfboard
tennis racket
bottle
wine glass
cup
fork
knife
spoon
bowl
banana
apple
sandwich
orange
broccoli
carrot
hot dog
pizza
donut
cake
chair
sofa
pottedplant
bed
diningtable
toilet
tvmonitor
laptop
mouse
remote
keyboard
cell phone
microwave
oven
toaster
sink
refrigerator
book
clock
vase
scissors
teddy bear
hair drier
toothbrush

그러니까..

결국에는 최소 5개의 파일이 필요한건가?

cfg, weight, data, name 그리고 테스트 이미지

pooling  - overfitting 방지

[링크 : https://hobinjeong.medium.com/cnn에서-pooling이란-c4e01aa83c83]

[링크 : http://hobinjeong.medium.com/cnn-convolutional-neural-network-9f600dd3b66395]

정리 잘된 동영상이 있어서 링크

convolution은 특정 신호에 반응하고 이미지 어디에 있는지를 확인하고

pooling은 위치나 각도에 둔감해지도록 인식율을 올리는 효과를 지니는 연산(?)

[링크 : https://www.youtube.com/watch?v=u0eT7VZAgRw]

RNN(Recurrent Neural Network)

CNN(Convolution Neural Network)

합성곱신경망, convolution과 pooling

[링크 : http://ebbnflow.tistory.com/119]

[링크 : http://dbrang.tistory.com/1537]

원본 darknet은 성능에 영향을 줄게 3개 밖에 없는데

alexeyAB의 darknet은 gpu, cudnn, avx, openmp 4가지 이다.

[링크 : https://github.com/AlexeyAB/darknet]

심심해서(?) i5-2세대도 있나 보는데 어라..? 있네?

[링크 : https://ark.intel.com/.../intel-core-i5-2500-processor-6m-cache-up-to-3-70-ghz.html]

[링크 : https://ark.intel.com/../intel-core-i5-2520m-processor-3m-cache-up-to-3-20-ghz.html]

근데 빌드해서 돌려보니 내꺼는 AVX일뿐이라 돌아가지 않는다 ㅠㅠ

AVX2는 하스웰 이후부터 지원한다고 하니.. 집에있는 내 실험용 컴퓨터로는 무리겠구나..

$ ./darknet detect cfg/yolov3.cfg ../yolov3.weights data/dog.jpg
 GPU isn't used 
 Used AVX 
 Not used FMA & AVX2 
 OpenCV isn't used - data augmentation will be slow 
명령어가 잘못됨 (core dumped)

alexeyAB 버전을 싱글 코어 / openmp 설정으로 돌리니 반정도 줄었다.

data/dog.jpg: Predicted in 11175.292000 milli-seconds.
data/dog.jpg: Predicted in 5974.575000 milli-seconds.

위는 오리지널 darknet을 아무런 옵션없이 라즈베리에서 빌드한 결과

$ ldd darknet
        linux-vdso.so.1 (0x7efe1000)
        /usr/lib/arm-linux-gnueabihf/libarmmem-${PLATFORM}.so => /usr/lib/arm-linux-gnueabihf/libarmmem-v7l.so (0x76f54000)
        libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x76eb6000)
        libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x76e8c000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76d3e000)
        /lib/ld-linux-armhf.so.3 (0x76f69000)

아래는 darknet AlexeyAB 버전을 neon과 openmp 설정해서 빌드한 결과

$ ldd darknet
        linux-vdso.so.1 (0x7eefc000)
        /usr/lib/arm-linux-gnueabihf/libarmmem-${PLATFORM}.so => /usr/lib/arm-linux-gnueabihf/libarmmem-v7l.so (0x76f79000)
        libgomp.so.1 => /lib/arm-linux-gnueabihf/libgomp.so.1 (0x76f25000)
        libstdc++.so.6 => /lib/arm-linux-gnueabihf/libstdc++.so.6 (0x76dde000)
        libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x76d5c000)
        libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x76d2f000)
        libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x76d05000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76bb7000)
        libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x76ba4000)
        /lib/ld-linux-armhf.so.3 (0x76f8e000)

+

cpu only = 30.13sec

$ ./tiny.sh
 GPU isn't used
 OpenCV isn't used - data augmentation will be slow
mini_batch = 1, batch = 1, time_steps = 1, train = 0
   layer   filters  size/strd(dil)      input                output
   0 conv     16       3 x 3/ 1    416 x 416 x   3 ->  416 x 416 x  16 0.150 BF
   1 max                2x 2/ 2    416 x 416 x  16 ->  208 x 208 x  16 0.003 BF
   2 conv     32       3 x 3/ 1    208 x 208 x  16 ->  208 x 208 x  32 0.399 BF
   3 max                2x 2/ 2    208 x 208 x  32 ->  104 x 104 x  32 0.001 BF
   4 conv     64       3 x 3/ 1    104 x 104 x  32 ->  104 x 104 x  64 0.399 BF
   5 max                2x 2/ 2    104 x 104 x  64 ->   52 x  52 x  64 0.001 BF
   6 conv    128       3 x 3/ 1     52 x  52 x  64 ->   52 x  52 x 128 0.399 BF
   7 max                2x 2/ 2     52 x  52 x 128 ->   26 x  26 x 128 0.000 BF
   8 conv    256       3 x 3/ 1     26 x  26 x 128 ->   26 x  26 x 256 0.399 BF
   9 max                2x 2/ 2     26 x  26 x 256 ->   13 x  13 x 256 0.000 BF
  10 conv    512       3 x 3/ 1     13 x  13 x 256 ->   13 x  13 x 512 0.399 BF
  11 max                2x 2/ 1     13 x  13 x 512 ->   13 x  13 x 512 0.000 BF
  12 conv   1024       3 x 3/ 1     13 x  13 x 512 ->   13 x  13 x1024 1.595 BF
  13 conv    256       1 x 1/ 1     13 x  13 x1024 ->   13 x  13 x 256 0.089 BF
  14 conv    512       3 x 3/ 1     13 x  13 x 256 ->   13 x  13 x 512 0.399 BF
  15 conv    255       1 x 1/ 1     13 x  13 x 512 ->   13 x  13 x 255 0.044 BF
  16 yolo
[yolo] params: iou loss: mse (2), iou_norm: 0.75, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.00
  17 route  13                                     ->   13 x  13 x 256
  18 conv    128       1 x 1/ 1     13 x  13 x 256 ->   13 x  13 x 128 0.011 BF
  19 upsample                 2x    13 x  13 x 128 ->   26 x  26 x 128
  20 route  19 8                                   ->   26 x  26 x 384
  21 conv    256       3 x 3/ 1     26 x  26 x 384 ->   26 x  26 x 256 1.196 BF
  22 conv    255       1 x 1/ 1     26 x  26 x 256 ->   26 x  26 x 255 0.088 BF
  23 yolo
[yolo] params: iou loss: mse (2), iou_norm: 0.75, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.00
Total BFLOPS 5.571
avg_outputs = 341534
Loading weights from ../yolov3-tiny.weights...
 seen 64, trained: 32013 K-images (500 Kilo-batches_64)
Done! Loaded 24 layers from weights-file
 Detection layer: 16 - type = 28
 Detection layer: 23 - type = 28
data/dog.jpg: Predicted in 30133.750000 milli-seconds.
dog: 81%
bicycle: 38%
car: 71%
truck: 42%
truck: 62%
car: 40%
Not compiled with OpenCV, saving to predictions.png instead

neon = 10.718 sec

$ ./tiny.sh
 GPU isn't used
 OpenCV isn't used - data augmentation will be slow
mini_batch = 1, batch = 1, time_steps = 1, train = 0
   layer   filters  size/strd(dil)      input                output
   0 conv     16       3 x 3/ 1    416 x 416 x   3 ->  416 x 416 x  16 0.150 BF
   1 max                2x 2/ 2    416 x 416 x  16 ->  208 x 208 x  16 0.003 BF
   2 conv     32       3 x 3/ 1    208 x 208 x  16 ->  208 x 208 x  32 0.399 BF
   3 max                2x 2/ 2    208 x 208 x  32 ->  104 x 104 x  32 0.001 BF
   4 conv     64       3 x 3/ 1    104 x 104 x  32 ->  104 x 104 x  64 0.399 BF
   5 max                2x 2/ 2    104 x 104 x  64 ->   52 x  52 x  64 0.001 BF
   6 conv    128       3 x 3/ 1     52 x  52 x  64 ->   52 x  52 x 128 0.399 BF
   7 max                2x 2/ 2     52 x  52 x 128 ->   26 x  26 x 128 0.000 BF
   8 conv    256       3 x 3/ 1     26 x  26 x 128 ->   26 x  26 x 256 0.399 BF
   9 max                2x 2/ 2     26 x  26 x 256 ->   13 x  13 x 256 0.000 BF
  10 conv    512       3 x 3/ 1     13 x  13 x 256 ->   13 x  13 x 512 0.399 BF
  11 max                2x 2/ 1     13 x  13 x 512 ->   13 x  13 x 512 0.000 BF
  12 conv   1024       3 x 3/ 1     13 x  13 x 512 ->   13 x  13 x1024 1.595 BF
  13 conv    256       1 x 1/ 1     13 x  13 x1024 ->   13 x  13 x 256 0.089 BF
  14 conv    512       3 x 3/ 1     13 x  13 x 256 ->   13 x  13 x 512 0.399 BF
  15 conv    255       1 x 1/ 1     13 x  13 x 512 ->   13 x  13 x 255 0.044 BF
  16 yolo
[yolo] params: iou loss: mse (2), iou_norm: 0.75, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.00
  17 route  13                                     ->   13 x  13 x 256
  18 conv    128       1 x 1/ 1     13 x  13 x 256 ->   13 x  13 x 128 0.011 BF
  19 upsample                 2x    13 x  13 x 128 ->   26 x  26 x 128
  20 route  19 8                                   ->   26 x  26 x 384
  21 conv    256       3 x 3/ 1     26 x  26 x 384 ->   26 x  26 x 256 1.196 BF
  22 conv    255       1 x 1/ 1     26 x  26 x 256 ->   26 x  26 x 255 0.088 BF
  23 yolo
[yolo] params: iou loss: mse (2), iou_norm: 0.75, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.00
Total BFLOPS 5.571
avg_outputs = 341534
Loading weights from ../yolov3-tiny.weights...
 seen 64, trained: 32013 K-images (500 Kilo-batches_64)
Done! Loaded 24 layers from weights-file
 Detection layer: 16 - type = 28
 Detection layer: 23 - type = 28
data/dog.jpg: Predicted in 10718.416000 milli-seconds.
dog: 81%
bicycle: 38%
car: 71%
truck: 42%
truck: 62%
car: 40%
Not compiled with OpenCV, saving to predictions.png instead

openmp = 8.686 sec

$ ./tiny.sh
 GPU isn't used
 OpenCV isn't used - data augmentation will be slow
mini_batch = 1, batch = 1, time_steps = 1, train = 0
   layer   filters  size/strd(dil)      input                output
   0 conv     16       3 x 3/ 1    416 x 416 x   3 ->  416 x 416 x  16 0.150 BF
   1 max                2x 2/ 2    416 x 416 x  16 ->  208 x 208 x  16 0.003 BF
   2 conv     32       3 x 3/ 1    208 x 208 x  16 ->  208 x 208 x  32 0.399 BF
   3 max                2x 2/ 2    208 x 208 x  32 ->  104 x 104 x  32 0.001 BF
   4 conv     64       3 x 3/ 1    104 x 104 x  32 ->  104 x 104 x  64 0.399 BF
   5 max                2x 2/ 2    104 x 104 x  64 ->   52 x  52 x  64 0.001 BF
   6 conv    128       3 x 3/ 1     52 x  52 x  64 ->   52 x  52 x 128 0.399 BF
   7 max                2x 2/ 2     52 x  52 x 128 ->   26 x  26 x 128 0.000 BF
   8 conv    256       3 x 3/ 1     26 x  26 x 128 ->   26 x  26 x 256 0.399 BF
   9 max                2x 2/ 2     26 x  26 x 256 ->   13 x  13 x 256 0.000 BF
  10 conv    512       3 x 3/ 1     13 x  13 x 256 ->   13 x  13 x 512 0.399 BF
  11 max                2x 2/ 1     13 x  13 x 512 ->   13 x  13 x 512 0.000 BF
  12 conv   1024       3 x 3/ 1     13 x  13 x 512 ->   13 x  13 x1024 1.595 BF
  13 conv    256       1 x 1/ 1     13 x  13 x1024 ->   13 x  13 x 256 0.089 BF
  14 conv    512       3 x 3/ 1     13 x  13 x 256 ->   13 x  13 x 512 0.399 BF
  15 conv    255       1 x 1/ 1     13 x  13 x 512 ->   13 x  13 x 255 0.044 BF
  16 yolo
[yolo] params: iou loss: mse (2), iou_norm: 0.75, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.00
  17 route  13                                     ->   13 x  13 x 256
  18 conv    128       1 x 1/ 1     13 x  13 x 256 ->   13 x  13 x 128 0.011 BF
  19 upsample                 2x    13 x  13 x 128 ->   26 x  26 x 128
  20 route  19 8                                   ->   26 x  26 x 384
  21 conv    256       3 x 3/ 1     26 x  26 x 384 ->   26 x  26 x 256 1.196 BF
  22 conv    255       1 x 1/ 1     26 x  26 x 256 ->   26 x  26 x 255 0.088 BF
  23 yolo
[yolo] params: iou loss: mse (2), iou_norm: 0.75, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.00
Total BFLOPS 5.571
avg_outputs = 341534
Loading weights from ../yolov3-tiny.weights...
 seen 64, trained: 32013 K-images (500 Kilo-batches_64)
Done! Loaded 24 layers from weights-file
 Detection layer: 16 - type = 28
 Detection layer: 23 - type = 28
data/dog.jpg: Predicted in 8686.237000 milli-seconds.
dog: 81%
bicycle: 38%
car: 71%
truck: 42%
truck: 62%
car: 40%
Not compiled with OpenCV, saving to predictions.png instead

openmp + neon = 4.449 sec

$ ./tiny.sh
 GPU isn't used
 OpenCV isn't used - data augmentation will be slow
mini_batch = 1, batch = 1, time_steps = 1, train = 0
   layer   filters  size/strd(dil)      input                output
   0 conv     16       3 x 3/ 1    416 x 416 x   3 ->  416 x 416 x  16 0.150 BF
   1 max                2x 2/ 2    416 x 416 x  16 ->  208 x 208 x  16 0.003 BF
   2 conv     32       3 x 3/ 1    208 x 208 x  16 ->  208 x 208 x  32 0.399 BF
   3 max                2x 2/ 2    208 x 208 x  32 ->  104 x 104 x  32 0.001 BF
   4 conv     64       3 x 3/ 1    104 x 104 x  32 ->  104 x 104 x  64 0.399 BF
   5 max                2x 2/ 2    104 x 104 x  64 ->   52 x  52 x  64 0.001 BF
   6 conv    128       3 x 3/ 1     52 x  52 x  64 ->   52 x  52 x 128 0.399 BF
   7 max                2x 2/ 2     52 x  52 x 128 ->   26 x  26 x 128 0.000 BF
   8 conv    256       3 x 3/ 1     26 x  26 x 128 ->   26 x  26 x 256 0.399 BF
   9 max                2x 2/ 2     26 x  26 x 256 ->   13 x  13 x 256 0.000 BF
  10 conv    512       3 x 3/ 1     13 x  13 x 256 ->   13 x  13 x 512 0.399 BF
  11 max                2x 2/ 1     13 x  13 x 512 ->   13 x  13 x 512 0.000 BF
  12 conv   1024       3 x 3/ 1     13 x  13 x 512 ->   13 x  13 x1024 1.595 BF
  13 conv    256       1 x 1/ 1     13 x  13 x1024 ->   13 x  13 x 256 0.089 BF
  14 conv    512       3 x 3/ 1     13 x  13 x 256 ->   13 x  13 x 512 0.399 BF
  15 conv    255       1 x 1/ 1     13 x  13 x 512 ->   13 x  13 x 255 0.044 BF
  16 yolo
[yolo] params: iou loss: mse (2), iou_norm: 0.75, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.00
  17 route  13                                     ->   13 x  13 x 256
  18 conv    128       1 x 1/ 1     13 x  13 x 256 ->   13 x  13 x 128 0.011 BF
  19 upsample                 2x    13 x  13 x 128 ->   26 x  26 x 128
  20 route  19 8                                   ->   26 x  26 x 384
  21 conv    256       3 x 3/ 1     26 x  26 x 384 ->   26 x  26 x 256 1.196 BF
  22 conv    255       1 x 1/ 1     26 x  26 x 256 ->   26 x  26 x 255 0.088 BF
  23 yolo
[yolo] params: iou loss: mse (2), iou_norm: 0.75, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.00
Total BFLOPS 5.571
avg_outputs = 341534
Loading weights from ../yolov3-tiny.weights...
 seen 64, trained: 32013 K-images (500 Kilo-batches_64)
Done! Loaded 24 layers from weights-file
 Detection layer: 16 - type = 28
 Detection layer: 23 - type = 28
data/dog.jpg: Predicted in 4449.888000 milli-seconds.
dog: 81%
bicycle: 38%
car: 71%
truck: 42%
truck: 62%
car: 40%
Not compiled with OpenCV, saving to predictions.png instead

원인 불명으로 BUS error나 Segmentation fault가 떠서 찾아보니

소스를 수정한 버전이 있다고 한다.

돌아는 가는데.... (이하 생략)

[링크 : https://j-remind.tistory.com/53]

[링크 : https://github.com/AlexeyAB/darknet]

설마 endian 문제?

[링크 : https://github.com/pjreddie/darknet/issues/823]

YOLOv2-tiny 의 web implementation 이라는데 웹 implementation이 머지?

[링크 : https://reu2018dl.github.io/]

[링크 : https://github.com/reu2018DL/YOLO-LITE]

[링크 : https://arxiv.org/pdf/1811.05588.pdf]

+

MS COCO 2014 and PASCAL VOC 2007 + 2012

pascal voc(Visual Object Classes)

[링크 : http://host.robots.ox.ac.uk/pascal/VOC/]

[링크 : https://ndb796.tistory.com/500]

MS COCO

COCO랑은 다른가?

[링크 : https://arxiv.org/pdf/1405.0312.pdf]

[링크 : https://chacha95.github.io/2020-02-27-Object-Detection4/]

[링크 : https://cocodataset.org/#home]

AP - Average Precision

mAP (mean AP)

[링크 : https://mezzaninex.tistory.com/entry/AI-COCO-Dataset-mAPmean-Average-Precision]

SSD 도 모델이군..

[링크 : https://pjreddie.com/darknet/yolo/]

[링크 : https://junjiwon1031.github.io/2017/09/08/Single-Shot-Multibox-Detector.html]