Validation notebook for ArcFace models




Use this notebook to verify the accuracy of a trained ArcFace model in ONNX format on the validation datasets.

Models supported

  • LResNet100E-IR (ResNet100 backend with ArcFace loss)



The following packages need to be installed before proceeding:

  • Protobuf compiler - sudo apt-get install protobuf-compiler libprotoc-dev (required for ONNX. This will work for any linux system. For detailed installation guidelines head over to ONNX documentation)
  • ONNX - pip install onnx
  • MXNet - pip install mxnet-cu90mkl --pre -U (tested on this version GPU, can use other versions. --pre indicates a pre build of MXNet which is required here for ONNX version compatibility. -U uninstalls any existing MXNet version allowing for a clean install)
  • numpy - pip install numpy
  • matplotlib - pip install matplotlib
  • OpenCV - pip install opencv-python
  • Scikit-learn - pip install scikit-learn
  • EasyDict - pip install easydict
  • pickle - pip install pickle

In order to do inference with a python script:

  • Generate the script : In Jupyter Notebook browser, go to File -> Download as -> Python (.py)
  • Run the script: python

Import dependencies


Verify that all dependencies are installed using the cell below. Continue if no errors encountered, warnings can be ignored.

Data loading helper code

  • load_bin() loads validation datasets
  • load_property() loads information like image size, number of identities from the property file in the dataset folder

Evaluation helper code

  • class LFold is used to split the data for K-fold crossvalidation
  • calculate_roc() computes accuracy on each fold of cross validation using calculate_accuracy()
  • calculate_accuracy() computes the actual accuracy for test samples by thresholding the distance between embedding vectors of a test image pair and comparing the output with the ground truth for the pair
  • evaluate() splits embeddings into test pairs and computes accuracies using calculate_roc()

Define test function


test() takes a validation set data_set and the MXNet model mx_model as input and computes accuracies using evaluate() on the set using nfolds cross validation.

Specify paths and parameters


Compute validation accuracies


This is the main function and it loads the ONNX model, loads the validation datasets and computes the validation accuracies on the data according to the verification targets specified above. The accuracy for each target is displayed in the output of the cell.