어디까지 흘러가야 답을 찾을수 있을까 ㅠㅠ

 

아래 사이트에서 pb와 tflite를 모두 제공해주어서 한번 시도

[링크 : https://tfhub.dev/google/aiy/vision/classifier/birds_V1/1]

 

위 링크의 pb를 확인해보면 아래와 같이 나오는데, tag-set이 'serve' 라는게 없어서 변환하려니 안된다

$ saved_model_cli show --dir ./ --all
2021-02-02 16:42:12.474893: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-02-02 16:42:12.474941: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

MetaGraphDef with tag-set: '' contains the following SignatureDefs:

signature_def['default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['images'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 224, 224, 3)
        name: hub_input/images:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['default'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 965)
        name: prediction:0
  Method name is:

signature_def['image_classifier']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['images'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 224, 224, 3)
        name: hub_input/images:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['logits'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 965)
        name: prediction:0
  Method name is:

 

이건 다른 모델.. 도대체 어떻게 봐야 하는걸까 ㅠㅠ

$ saved_model_cli show --dir ./ --all
2021-02-02 16:42:44.443703: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-02-02 16:42:44.443752: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['__saved_model_init_op']:
  The given SavedModel SignatureDef contains the following input(s):
  The given SavedModel SignatureDef contains the following output(s):
    outputs['__saved_model_init_op'] tensor_info:
        dtype: DT_INVALID
        shape: unknown_rank
        name: NoOp
  Method name is:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['input_tensor'] tensor_info:
        dtype: DT_UINT8
        shape: (1, -1, -1, 3)
        name: serving_default_input_tensor:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['detection_anchor_indices'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 100)
        name: StatefulPartitionedCall:0
    outputs['detection_boxes'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 100, 4)
        name: StatefulPartitionedCall:1
    outputs['detection_classes'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 100)
        name: StatefulPartitionedCall:2
    outputs['detection_multiclass_scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 100, 91)
        name: StatefulPartitionedCall:3
    outputs['detection_scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 100)
        name: StatefulPartitionedCall:4
    outputs['num_detections'] tensor_info:
        dtype: DT_FLOAT
        shape: (1)
        name: StatefulPartitionedCall:5
    outputs['raw_detection_boxes'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 1917, 4)
        name: StatefulPartitionedCall:6
    outputs['raw_detection_scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 1917, 91)
        name: StatefulPartitionedCall:7
  Method name is: tensorflow/serving/predict

Defined Functions:
  Function Name: '__call__'
    Option #1
      Callable with:
        Argument #1
          input_tensor: TensorSpec(shape=(1, None, None, 3), dtype=tf.uint8, name='input_tensor')

 

혹시나 해서 다시 한번 select_tf_ops 옵션을 빼고 해보았지만 역시나 안된다.

눈에 들어오는 에러는 아래것 정도인데 -emit-select-tf-ops 옵션을 누구에게 주어야 하는건지 모르겠다.

그리고 custom op nor flex op. flex op는 또 무엇인가...

tensorflow.lite.python.convert.ConverterError: <unknown>:0: error: loc(callsite(callsite("Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField_1/Size@__inference___call___21591" at "StatefulPartitionedCall@__inference_signature_wrapper_23250") at "StatefulPartitionedCall")): 'tf.Size' op is neither a custom op nor a flex op
<unknown>:0: note: loc("StatefulPartitionedCall"): called from
<unknown>:0: error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag):
        tf.Size {device = ""}

 

 

$ cat c.py
import tensorflow as tf

saved_model_dir="./"
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
#converter.target_spec.supported_ops = [
#  tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
#  tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
#]
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)

$ python3 c.py
2021-02-02 16:45:08.234068: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-02-02 16:45:08.234112: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-02-02 16:45:10.382490: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-02-02 16:45:10.382710: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-02-02 16:45:10.382742: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2021-02-02 16:45:10.382775: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (mini2760p): /proc/driver/nvidia/version does not exist
2021-02-02 16:45:10.383265: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-02-02 16:45:26.331917: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:316] Ignored output_format.
2021-02-02 16:45:26.331970: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:319] Ignored drop_control_dependency.
2021-02-02 16:45:26.331981: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:325] Ignored change_concat_input_ranges.
2021-02-02 16:45:26.333113: I tensorflow/cc/saved_model/reader.cc:32] Reading SavedModel from: ./
2021-02-02 16:45:26.442652: I tensorflow/cc/saved_model/reader.cc:55] Reading meta graph with tags { serve }
2021-02-02 16:45:26.442721: I tensorflow/cc/saved_model/reader.cc:93] Reading SavedModel debug info (if present) from: ./
2021-02-02 16:45:26.442798: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-02-02 16:45:26.752919: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2021-02-02 16:45:26.824027: I tensorflow/cc/saved_model/loader.cc:206] Restoring SavedModel bundle.
2021-02-02 16:45:26.900734: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2494085000 Hz
2021-02-02 16:45:27.788741: I tensorflow/cc/saved_model/loader.cc:190] Running initialization op on SavedModel bundle at path: ./
2021-02-02 16:45:28.227404: I tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: success: OK. Took 1894293 microseconds.
2021-02-02 16:45:34.080047: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:194] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2021-02-02 16:45:35.369335: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
loc(callsite(callsite("Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField_1/Size@__inference___call___21591" at "StatefulPartitionedCall@__inference_signature_wrapper_23250") at "StatefulPartitionedCall")): error: 'tf.Size' op is neither a custom op nor a flex op
error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag):
        tf.Size {device = ""}
Traceback (most recent call last):
  File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/convert.py", line 210, in toco_convert_protos
    model_str = wrap_toco.wrapped_toco_convert(model_flags_str,
  File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/wrap_toco.py", line 32, in wrapped_toco_convert
    return _pywrap_toco_api.TocoConvert(
Exception: <unknown>:0: error: loc(callsite(callsite("Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField_1/Size@__inference___call___21591" at "StatefulPartitionedCall@__inference_signature_wrapper_23250") at "StatefulPartitionedCall")): 'tf.Size' op is neither a custom op nor a flex op
<unknown>:0: note: loc("StatefulPartitionedCall"): called from
<unknown>:0: error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag):
        tf.Size {device = ""}


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c.py", line 9, in <module>
    tflite_model = converter.convert()
  File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/lite.py", line 739, in convert
    result = _convert_saved_model(**converter_kwargs)
  File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/convert.py", line 632, in convert_saved_model
    data = toco_convert_protos(
  File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/convert.py", line 216, in toco_convert_protos
    raise ConverterError(str(e))
tensorflow.lite.python.convert.ConverterError: <unknown>:0: error: loc(callsite(callsite("Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField_1/Size@__inference___call___21591" at "StatefulPartitionedCall@__inference_signature_wrapper_23250") at "StatefulPartitionedCall")): 'tf.Size' op is neither a custom op nor a flex op
<unknown>:0: note: loc("StatefulPartitionedCall"): called from
<unknown>:0: error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag):
        tf.Size {device = ""}

 

saved_model_cli 명령어를 이용해서 변환하려는데 tensorrt가 안들어가면 인자가 부족하다고 하고

넣으면 libvinfer 에러가 나고... 후...

$ saved_model_cli convert --dir=. --output_dir=output --tag_set serving_default tensorrt
2021-02-02 16:52:12.317957: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-02-02 16:52:12.318003: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-02-02 16:52:13.640651: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2021-02-02 16:52:13.640699: F tensorflow/compiler/tf2tensorrt/stub/nvinfer_stub.cc:49] getInferLibVersion symbol not found.
중지됨 (core dumped)

[링크 : https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/saved_model_cli.py]

 

-emi-select-tf-ops=true가 flatbuffer_translate 에 전달되는 옵션인가?

// RUN: flatbuffer_translate -mlir-to-tflite-flatbuffer %s -emit-select-tf-ops=true -emit-builtin-tflite-ops=false -o - | flatbuffer_to_string - | FileCheck %s

[링크 : http://110.249.209.116/tiansongzhao/QT-Platform/-/blob/ffe4404132bbba3c690232c9f846ac160aa38e65/Software/resource/samples/tensorflow/tensorflow/compiler/mlir/lite/tests/mlir2flatbuffer/flex_exclusively.mlir]

 

MLIR (Multi-Level Intermediate Representation)

[링크 : https://mlir.llvm.org/]

 

결국은 돌아돌아 원점인가..

import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.target_spec.supported_ops = [
  tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
  tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)

[링크 : https://www.tensorflow.org/lite/guide/ops_select#convert_a_model]

 

TFLite interpreter가 select ops 를 가지고 있는지 확인해라..

tensorflow.a 가 그럼 그 옵션을 받아야 한다는건가?

Try using TF select ops. However, you may needs to ensure your TFLite interpreter has these select ops for inference.

  [링크 : https://stackoverflow.com/questions/65661737/]

 

'프로그램 사용 > yolo_tensorflow_golo' 카테고리의 다른 글

tf object detection COCO  (0) 2021.02.05
tensorflow lite SELECT_TF_OPS  (0) 2021.02.02
saved_model_cli  (0) 2021.02.02
tensorflow bazel build 옵션  (0) 2021.02.02
tensorflow bazel build  (0) 2021.02.01
convert from tensorflow to tensorflow lite  (0) 2021.02.01
Posted by 구차니

댓글을 달아 주세요