Restart.yml file cannot be read in due to ConstructorError

[amarkpayne]

I was running a large ARC job that crashed due to an unrelated and now fixed issue, and I tried to restart from the restart file generated by ARC previously. However, it appears that something is not getting saved correctly to the restart file. Below is the Traceback (I have edited to reduce the lengths of file paths, but I checked that everything was being called from the correct place).

Traceback (most recent call last):
  File "ARC/ARC.py", line 73, in <module>
    main()
  File "ARC/ARC.py", line 53, in main
    input_dict = read_yaml_file(input_file)
  File "ARC/arc/common.py", line 265, in read_yaml_file
    content = yaml.load(stream=f, Loader=yaml.FullLoader)
  File "anaconda3/envs/arc_env/lib/python3.7/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "anaconda3/envs/arc_env/lib/python3.7/site-packages/yaml/constructor.py", line 43, in get_single_data
    return self.construct_document(node)
  File "anaconda3/envs/arc_env/lib/python3.7/site-packages/yaml/constructor.py", line 52, in construct_document
    for dummy in generator:
  File "anaconda3/envs/arc_env/lib/python3.7/site-packages/yaml/constructor.py", line 399, in construct_yaml_seq
    data.extend(self.construct_sequence(node))
  File "anaconda3/envs/arc_env/lib/python3.7/site-packages/yaml/constructor.py", line 122, in construct_sequence
    for child in node.value]
  File "anaconda3/envs/arc_env/lib/python3.7/site-packages/yaml/constructor.py", line 122, in <listcomp>
    for child in node.value]
  File "anaconda3/envs/arc_env/lib/python3.7/site-packages/yaml/constructor.py", line 94, in construct_object
    data = constructor(self, tag_suffix, node)
  File "anaconda3/envs/arc_env/lib/python3.7/site-packages/yaml/constructor.py", line 624, in construct_python_object_apply
    instance = self.make_python_instance(suffix, node, args, kwds, newobj)
  File "anaconda3/envs/arc_env/lib/python3.7/site-packages/yaml/constructor.py", line 570, in make_python_instance
    node.start_mark)
yaml.constructor.ConstructorError: while constructing a Python instance
expected a class, but found <class 'builtin_function_or_method'>
  in "restart.yml", line 9491, column 5

The relevant lines from the restart file are given below:

  mol: |
    1  C u0 p0 c0 {2,D} {6,S} {8,S}
    2  C u0 p0 c0 {1,D} {3,S} {9,S}
    3  C u0 p0 c0 {2,S} {4,D} {10,S}
    4  C u0 p0 c0 {3,D} {5,S} {7,S}
    5  C u0 p0 c0 {4,S} {6,D} {11,S}
    6  C u0 p0 c0 {1,S} {5,D} {12,S}
    7  C u0 p0 c0 {4,S} {13,S} {14,S} {15,S}
    8  H u0 p0 c0 {1,S}
    9  H u0 p0 c0 {2,S}
    10 H u0 p0 c0 {3,S}
    11 H u0 p0 c0 {5,S}
    12 H u0 p0 c0 {6,S}
    13 H u0 p0 c0 {7,S}
    14 H u0 p0 c0 {7,S}
    15 H u0 p0 c0 {7,S}
  most_stable_conformer: 0
  multiplicity: 1
  neg_freqs_trshed:
  - !!python/object/apply:numpy.core.multiarray.scalar  # This is line 9491
    - &id002 !!python/object/apply:numpy.dtype
      args:
      - f8
      - 0
      - 1
      state: !!python/tuple
      - 3
      - <
      - null
      - null
      - null
      - -1
      - -1
      - 0
    - !!binary |
      7FG4HoVrOMA=
  number_of_rotors: 0
  opt_level: b3lyp/def2tzvp empiricaldispersion=gd3bj
  optical_isomers: null
  t1: null

Is this an ARC issue or a YAML issue?

[amarkpayne]

Potentially relevant fix: facebookresearch/Detectron#840

[amarkpayne]

I can confirm that changing

ARC/arc/common.py

Lines 266 to 268 in 8242bba

	with open(path, 'r') as f:
	content = yaml.load(stream=f, Loader=yaml.FullLoader)
	return content

so that it now executes content = yaml.load(stream=f, Loader=yaml.Loader) appears to have solved my issue. @alongd is there any reason why you specified yaml.FullLoader here? In the meantime I'll look at PyYaml to see if they are aware of this issue and if they have a better fix, as well as try to learn more about what the differences are between these two loaders.

[alongd]

related to #223

[alongd]

from PyYAML:
https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation

The current Loader choices are:

• BaseLoader
  Only loads the most basic YAML

• SafeLoader
  Loads a subset of the YAML language, safely. This is recommended for loading untrusted input.

• FullLoader
  Loads the full YAML language. Avoids arbitrary code execution. This is currently (PyYAML 5.1) the default loader called by yaml.load(input) (after issuing the warning).

• UnsafeLoader (also called Loader for backwards compatability)
  The original Loader code that could be easily exploitable by untrusted data input.

So yaml.Loader is yaml.UnsafeLoader which could easily be exploited.

I think the fix here should be converting any numpy arrays saved to the restart file to regular arrays, if we really need them, or getting rid of them.

[amarkpayne]

related to #223

Thanks for the catch! I searched the open and closed issues before opening this one, but I forgot to check PRs as well. I think your proposed solution works as well.