프로그램 사용/yolo_tensorflow2024. 1. 10. 15:30

ssd-mobilenetv2 on jupyter notebook

버전 정보 추적

 

python 3.7

keras 2.8.0

numpy 1.21.6

[링크 : https://github.com/saunack/MobileNetv2-SSD/blob/master/model.ipynb]

 

설치한 버전들

$ pip install tensorflow==2.8.0
$ pip install numpy==1.21.6
$ pip install keras==2.8.0
$ pip install protobuf==3.19.0

 

------

keras 2.8을 써야 한다고 나오니 keras의 릴리즈 날짜로 추적

v2.8.0
 on Jan 7, 2022  d8fcb9d  zip  tar.gz  Notes

[링크 : https://github.com/keras-team/keras/tags?after=v2.9.0-rc1]

 

tensorflow 버전 추적

v2.8.0
 on Feb 1, 2022  3f878cf  zip  tar.gz  Notes

[링크 : https://github.com/tensorflow/tensorflow/tags?after=v2.7.2]

 

protobuf 3.19.0

numpy 1.24.4 (1.25 미만)

$ pip install numpy==1.34
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement numpy==1.34 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6, 1.22.0, 1.22.1, 1.22.2, 1.22.3, 1.22.4, 1.23.0rc1, 1.23.0rc2, 1.23.0rc3, 1.23.0, 1.23.1, 1.23.2, 1.23.3, 1.23.4, 1.23.5, 1.24.0rc1, 1.24.0rc2, 1.24.0, 1.24.1, 1.24.2, 1.24.3, 1.24.4, 1.25.0rc1, 1.25.0, 1.25.1, 1.25.2, 1.26.0b1, 1.26.0rc1, 1.26.0, 1.26.1, 1.26.2, 1.26.3)
ERROR: No matching distribution found for numpy==1.34

------

 

으아아아아 tensorflow가 문제냐 keras가 문제냐 ㅠㅠ

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

tf_v2 끄고 하면 아래와 같이 나오고

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[27], line 1
----> 1 history = model.fit(train_dataset,
      2                     epochs=25,
      3                     validation_data = test_dataset,
      4                     validation_steps=1)

File ~/.local/lib/python3.10/site-packages/keras/engine/training_v1.py:773, in Model.fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
    771 if kwargs:
    772   raise TypeError('Unrecognized keyword arguments: ' + str(kwargs))
--> 773 self._assert_compile_was_called()
    774 self._check_call_args('fit')
    776 func = self._select_training_loop(x)

File ~/.local/lib/python3.10/site-packages/keras/engine/training_v1.py:2788, in Model._assert_compile_was_called(self)
   2782 def _assert_compile_was_called(self):
   2783   # Checks whether `compile` has been called. If it has been called,
   2784   # then the optimizer is set. This is different from whether the
   2785   # model is compiled
   2786   # (i.e. whether the model is built and its inputs/outputs are set).
   2787   if not self._compile_was_called:
-> 2788     raise RuntimeError('You must compile your model before '
   2789                        'training/testing. '
   2790                        'Use `model.compile(optimizer, loss)`.')

RuntimeError: You must compile your model before training/testing. Use `model.compile(optimizer, loss)`.

 

tf_v2를 쓰게 하면

import tensorflow.compat.v1 as tf
#tf.disable_v2_behavior()

사용자 쪽 코드로 문제를 넘기는데 머가 문제일까..

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[28], line 1
----> 1 history = model.fit(train_dataset,
      2                     epochs=25,
      3                     validation_data = test_dataset,
      4                     validation_steps=1)

File ~/.local/lib/python3.10/site-packages/keras/engine/training_v1.py:777, in Model.fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
    774 self._check_call_args('fit')
    776 func = self._select_training_loop(x)
--> 777 return func.fit(
    778     self,
    779     x=x,
    780     y=y,
    781     batch_size=batch_size,
    782     epochs=epochs,
    783     verbose=verbose,
    784     callbacks=callbacks,
    785     validation_split=validation_split,
    786     validation_data=validation_data,
    787     shuffle=shuffle,
    788     class_weight=class_weight,
    789     sample_weight=sample_weight,
    790     initial_epoch=initial_epoch,
    791     steps_per_epoch=steps_per_epoch,
    792     validation_steps=validation_steps,
    793     validation_freq=validation_freq,
    794     max_queue_size=max_queue_size,
    795     workers=workers,
    796     use_multiprocessing=use_multiprocessing)

File ~/.local/lib/python3.10/site-packages/keras/engine/training_arrays_v1.py:616, in ArrayLikeTrainingLoop.fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, **kwargs)
    595 def fit(self,
    596         model,
    597         x=None,
   (...)
    611         validation_freq=1,
    612         **kwargs):
    613   batch_size = model._validate_or_infer_batch_size(batch_size,
    614                                                    steps_per_epoch, x)
--> 616   x, y, sample_weights = model._standardize_user_data(
    617       x,
    618       y,
    619       sample_weight=sample_weight,
    620       class_weight=class_weight,
    621       batch_size=batch_size,
    622       check_steps=True,
    623       steps_name='steps_per_epoch',
    624       steps=steps_per_epoch,
    625       validation_split=validation_split,
    626       shuffle=shuffle)
    628   if validation_data:
    629     val_x, val_y, val_sample_weights = model._prepare_validation_data(
    630         validation_data, batch_size, validation_steps)

File ~/.local/lib/python3.10/site-packages/keras/engine/training_v1.py:2318, in Model._standardize_user_data(self, x, y, sample_weight, class_weight, batch_size, check_steps, steps_name, steps, validation_split, shuffle, extract_tensors_from_dataset)
   2316 is_compile_called = False
   2317 if not self._is_compiled and self.optimizer:
-> 2318   self._compile_from_inputs(all_inputs, y_input, x, y)
   2319   is_compile_called = True
   2321 # In graph mode, if we had just set inputs and targets as symbolic tensors
   2322 # by invoking build and compile on the model respectively, we do not have to
   2323 # feed anything to the model. Model already has input and target data as
   (...)
   2327 
   2328 # self.run_eagerly is not free to compute, so we want to reuse the value.

File ~/.local/lib/python3.10/site-packages/keras/engine/training_v1.py:2568, in Model._compile_from_inputs(self, all_inputs, target, orig_inputs, orig_target)
   2565   else:
   2566     target_tensors = None
-> 2568 self.compile(
   2569     optimizer=self.optimizer,
   2570     loss=self.loss,
   2571     metrics=self._compile_metrics,
   2572     weighted_metrics=self._compile_weighted_metrics,
   2573     loss_weights=self.loss_weights,
   2574     target_tensors=target_tensors,
   2575     sample_weight_mode=self.sample_weight_mode,
   2576     run_eagerly=self.run_eagerly,
   2577     experimental_run_tf_function=self._experimental_run_tf_function)

File ~/.local/lib/python3.10/site-packages/tensorflow/python/training/tracking/base.py:629, in no_automatic_dependency_tracking.<locals>._method_wrapper(self, *args, **kwargs)
    627 self._self_setattr_tracking = False  # pylint: disable=protected-access
    628 try:
--> 629   result = method(self, *args, **kwargs)
    630 finally:
    631   self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

File ~/.local/lib/python3.10/site-packages/keras/engine/training_v1.py:443, in Model.compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, weighted_metrics, target_tensors, distribute, **kwargs)
    439 training_utils_v1.prepare_sample_weight_modes(
    440     self._training_endpoints, sample_weight_mode)
    442 # Creates the model loss and weighted metrics sub-graphs.
--> 443 self._compile_weights_loss_and_weighted_metrics()
    445 # Functions for train, test and predict will
    446 # be compiled lazily when required.
    447 # This saves time when the user is not using all functions.
    448 self.train_function = None

File ~/.local/lib/python3.10/site-packages/tensorflow/python/training/tracking/base.py:629, in no_automatic_dependency_tracking.<locals>._method_wrapper(self, *args, **kwargs)
    627 self._self_setattr_tracking = False  # pylint: disable=protected-access
    628 try:
--> 629   result = method(self, *args, **kwargs)
    630 finally:
    631   self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

File ~/.local/lib/python3.10/site-packages/keras/engine/training_v1.py:1537, in Model._compile_weights_loss_and_weighted_metrics(self, sample_weights)
   1524 self._handle_metrics(
   1525     self.outputs,
   1526     targets=self._targets,
   (...)
   1529     masks=masks,
   1530     return_weighted_metrics=True)
   1532 # Compute total loss.
   1533 # Used to keep track of the total loss value (stateless).
   1534 # eg., total_loss = loss_weight_1 * output_1_loss_fn(...) +
   1535 #                   loss_weight_2 * output_2_loss_fn(...) +
   1536 #                   layer losses.
-> 1537 self.total_loss = self._prepare_total_loss(masks)

File ~/.local/lib/python3.10/site-packages/keras/engine/training_v1.py:1597, in Model._prepare_total_loss(self, masks)
   1594     sample_weight *= mask
   1596 if hasattr(loss_fn, 'reduction'):
-> 1597   per_sample_losses = loss_fn.call(y_true, y_pred)
   1598   weighted_losses = losses_utils.compute_weighted_loss(
   1599       per_sample_losses,
   1600       sample_weight=sample_weight,
   1601       reduction=losses_utils.ReductionV2.NONE)
   1602   loss_reduction = loss_fn.reduction

File ~/.local/lib/python3.10/site-packages/keras/losses.py:245, in LossFunctionWrapper.call(self, y_true, y_pred)
    242   y_pred, y_true = losses_utils.squeeze_or_expand_dimensions(y_pred, y_true)
    244 ag_fn = tf.__internal__.autograph.tf_convert(self.fn, tf.__internal__.autograph.control_status_ctx())
--> 245 return ag_fn(y_true, y_pred, **self._fn_kwargs)

File ~/.local/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py:692, in convert.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    690 except Exception as e:  # pylint:disable=broad-except
    691   if hasattr(e, 'ag_error_metadata'):
--> 692     raise e.ag_error_metadata.to_exception(e)
    693   else:
    694     raise

ValueError: in user code:

    File "/tmp/ipykernel_49162/810674056.py", line 8, in Loss  *
        loss += confidenceLoss(y[:,:,:-4],tf.cast(gt[:,:,0],tf.int32))
    File "/tmp/ipykernel_49162/2037607510.py", line 2, in confidenceLoss  *
        unweighted_loss = tf.nn.sparse_softmax_cross_entropy_with_logits(label, y)

    ValueError: Only call sparse_softmax_cross_entropy_with_logits with named arguments (labels=..., logits=..., ...). Received unnamed argument: Tensor("loss/output_1_loss/Cast:0", shape=(None, None), dtype=int32)

 

버전이 문제였고 tensorflow는 그냥 v2로 쓰면 되는거였네 -_-

import tensorflow as tf
#import tensorflow.compat.v1 as tf
#tf.disable_v2_behavior()

 

최초는 2020년 7월 22일, 나중은 2022년 7월 21일(2년 만!)

saunack committed on Jul 21, 2022
saunack committed on Jul 22, 2020 

[링크 : https://github.com/saunack/MobileNetv2-SSD/commits/master/model.ipynb]

[링크 : https://github.com/saunack/MobileNetv2-SSD/blob/master/model.ipynb]

 

 

모델 저장(실패)

from keras.models import load_model
model.save('mnist_mlp_model.h5')

 

에러는 아래와 같이 나옴

NotImplementedError: Saving the model to HDF5 format requires the model to be a Functional model or a Sequential model. It does not work for subclassed models, because such models are defined via the body of a Python method, which isn't safely serializable. Consider saving to the Tensorflow SavedModel format (by setting save_format="tf") or using `save_weights`.

 

TensorFlow 2.0
TL;DR:

do not use model.save() for custom subclass keras model;
use save_weights() and load_weights() instead.

[링크 : https://stackoverflow.com/questions/51806852/cant-save-custom-subclassed-model]

 

sequential_model.save_weights("ckpt")

[링크 : https://www.tensorflow.org/guide/keras/save_and_serialize?hl=ko]

 

model.save_weights('model_weights', save_format='tf')

 

AttributeError: in user code:

    File "/home/user/.local/lib/python3.10/site-packages/keras/saving/saving_utils.py", line 138, in _wrapped_model  *
        outputs = model(*args, **kwargs)
    File "/tmp/ipykernel_53483/1508227539.py", line 46, in call  *
        x = self.MobileNet(x)
    File "/tmp/ipykernel_53483/3997091176.py", line 70, in call  *
        x = self.B2_2(x)
    File "/tmp/ipykernel_53483/1796771022.py", line 69, in call  *
        x = self.residual([inputs,x])
    File "/home/user/.local/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler  **
        raise e.with_traceback(filtered_tb) from None
    File "/home/user/.local/lib/python3.10/site-packages/keras/engine/base_layer.py", line 1102, in __call__
        if self._saved_model_inputs_spec is None:

    AttributeError: 'Add' object has no attribute '_saved_model_inputs_spec'

[링크 : https://github.com/tensorflow/tensorflow/issues/29545]

 

에라이 저장을 못하겠다!

[링크 : https://www.tensorflow.org/lite/convert?hl=ko]

 

엉뚱(?)한데서 터지는 느낌인데

tensorflow 버전을 2.14.0 으로 올려야 하나? 2.8.0이 아니라?

[링크 : https://www.tensorflow.org/api_docs/python/tf/keras/layers/Add]

 

+

24.01.11

2.14.0 으로 한다고 달라지는 건 없음.. 도대체 Add 객체는 멀까?

 

+

def SSD()  로 생성된걸 keras.Sequential로 감싸고 학습은 진행되는데.. 저장이 왜 또 안될까? ㅠㅠ

model = SSD(numBoxes=numBoxes, layerWidth=layerWidths, k = outputChannels)
model = tf.keras.Sequential(model)
# model.model().summary()

[링크 : https://www.tensorflow.org/tutorials/keras/save_and_load?hl=ko]

[링크 : https://www.tensorflow.org/hub/exporting_tf2_saved_model?hl=ko]

저작자표시

'프로그램 사용 > yolo_tensorflow' 카테고리의 다른 글

MobileNetV2 SSD FPN-Lite  (0) 2024.01.11
mobilenet v2 ssd  (0) 2024.01.11
텐서플로우 v1 을 v2로 마이그레이션은 실패 -_-  (0) 2024.01.10
골빈해커의 3분 딥러닝 github  (0) 2024.01.10
ReLU - Rectified Linear Unit  (0) 2024.01.10
Posted by 구차니

티스토리툴바