Error in imod.idf.open function from imod-python - Forum - iMOD
iMOD Forum
Error in imod.idf.open function from imod-python
Kristina Gumgowski, modified 3 Years ago.
Error in imod.idf.open function from imod-python
Youngling Posts: 13 Join Date: 3/18/21 Recent Posts
Hi,
I am experiencing an Error using the idf.open function from the imod python package. I get the following response:
Traceback (most recent call last):
File "C:/Users/Kristina/PycharmProjects/imod/main.py", line 43, in <module>
nnnn = im.idf.open("COMBINED_MAX_10_15.IDF")
File "C:\Users\Kristina\anaconda3\envs\imod\lib\site-packages\imod\idf.py", line 250, in open
return array_io.reading._load(paths, use_cftime, pattern, _read, header)
File "C:\Users\Kristina\anaconda3\envs\imod\lib\site-packages\imod\array_io\reading.py", line 348, in _load
headers = [header(p, pattern) for p in paths]
File "C:\Users\Kristina\anaconda3\envs\imod\lib\site-packages\imod\array_io\reading.py", line 348, in <listcomp>
headers = [header(p, pattern) for p in paths]
File "C:\Users\Kristina\anaconda3\envs\imod\lib\site-packages\imod\idf.py", line 46, in header
raise ValueError(f"Not a supported IDF file: {path}")
ValueError: Not a supported IDF file: COMBINED_MAX_10_15.IDF
I could not trace back the bug, the file I am trying to upload is working fine when I open it in the iMOD GUI. It seems to have something to do with the header but I do not know how to change it when creating the IDF.
Thanks in advance & best wishes
Kris
I am experiencing an Error using the idf.open function from the imod python package. I get the following response:
Traceback (most recent call last):
File "C:/Users/Kristina/PycharmProjects/imod/main.py", line 43, in <module>
nnnn = im.idf.open("COMBINED_MAX_10_15.IDF")
File "C:\Users\Kristina\anaconda3\envs\imod\lib\site-packages\imod\idf.py", line 250, in open
return array_io.reading._load(paths, use_cftime, pattern, _read, header)
File "C:\Users\Kristina\anaconda3\envs\imod\lib\site-packages\imod\array_io\reading.py", line 348, in _load
headers = [header(p, pattern) for p in paths]
File "C:\Users\Kristina\anaconda3\envs\imod\lib\site-packages\imod\array_io\reading.py", line 348, in <listcomp>
headers = [header(p, pattern) for p in paths]
File "C:\Users\Kristina\anaconda3\envs\imod\lib\site-packages\imod\idf.py", line 46, in header
raise ValueError(f"Not a supported IDF file: {path}")
ValueError: Not a supported IDF file: COMBINED_MAX_10_15.IDF
I could not trace back the bug, the file I am trying to upload is working fine when I open it in the iMOD GUI. It seems to have something to do with the header but I do not know how to change it when creating the IDF.
Thanks in advance & best wishes
Kris
Frans Roelofsen, modified 3 Years ago.
RE: Error in imod.idf.open function from imod-python
Padawan Posts: 51 Join Date: 11/11/13 Recent Posts
Hi Kristina,
My colleague of the iMOD Python development group suggested you probably use a former version of iMOD-Python. You can check your version with the command 'conda list imod' or 'pip show imod'. The version must be 0.10.1 or newer.
I you need an update, I normally use the command 'conda update -c conda-forge imod'. If conda will not update your iMOD version my colleague advices to do an iMOD-Python installation with the following environment: https://imod.xyz/installation.html#setting-up-an-environment
Kind regards,
Frans
My colleague of the iMOD Python development group suggested you probably use a former version of iMOD-Python. You can check your version with the command 'conda list imod' or 'pip show imod'. The version must be 0.10.1 or newer.
I you need an update, I normally use the command 'conda update -c conda-forge imod'. If conda will not update your iMOD version my colleague advices to do an iMOD-Python installation with the following environment: https://imod.xyz/installation.html#setting-up-an-environment
Kind regards,
Frans
Kristina Gumgowski, modified 3 Years ago.
RE: Error in imod.idf.open function from imod-python
Youngling Posts: 13 Join Date: 3/18/21 Recent Posts
Hi Frans,
I have problems to install imod using the development installation. With anaconda its not possible to install 0.10.1 or newer. When I do the development installation i get an error related to the geopandas package when running the script.
The final message is this one: "OSError: could not find or load spatialindex_c-64.dll"
From what I understood here ("https://github.com/geopandas/geopandas/issues/1812") it could be related to the mix of install channels or so. To be honest I am not deep enough into python to solve it but I also see that this forum is also not the place to ask these kind of questions either. Could you guide me where to ask instead?
Kind regards,
Kris
I have problems to install imod using the development installation. With anaconda its not possible to install 0.10.1 or newer. When I do the development installation i get an error related to the geopandas package when running the script.
The final message is this one: "OSError: could not find or load spatialindex_c-64.dll"
From what I understood here ("https://github.com/geopandas/geopandas/issues/1812") it could be related to the mix of install channels or so. To be honest I am not deep enough into python to solve it but I also see that this forum is also not the place to ask these kind of questions either. Could you guide me where to ask instead?
Kind regards,
Kris
Huite Bootsma, modified 3 Years ago.
RE: Error in imod.idf.open function from imod-python
Youngling Posts: 3 Join Date: 5/28/21 Recent Posts
Hello Kristina,
Apologies for late reply, Frans notified me, but I have to admit it slipped my attention unfortunately.
I actually see from the Github issue thread that another Deltares colleague ran into the issue recently:
https://github.com/Deltares/hydromt/issues/16
It does seem likely it has something to do with the mixing of environments. To force conda to use conda-forge, the following commands will do:
This might not fix your current environment, however. So if you still run into trouble, I'd suggest:
* Delete the current conda environment in which imod-python has been installed (conda env remove --name myenv)
* Reinstall the environment using the environment.yml as on the imod-python documentation
* Set the channel priorities as mentioned above
* Then do the development install
* Install other packages as needed, with the channel priorities set, it should go alright
We're also aware the previous release is relatively old (October 2020), I think I'll make a new release next week.
Apologies for late reply, Frans notified me, but I have to admit it slipped my attention unfortunately.
I actually see from the Github issue thread that another Deltares colleague ran into the issue recently:
https://github.com/Deltares/hydromt/issues/16
It does seem likely it has something to do with the mixing of environments. To force conda to use conda-forge, the following commands will do:
| 1 | |
| 2 | conda config --env --add channels conda-forge |
| 3 | conda config --env --set channel_priority strict |
This might not fix your current environment, however. So if you still run into trouble, I'd suggest:
* Delete the current conda environment in which imod-python has been installed (conda env remove --name myenv)
* Reinstall the environment using the environment.yml as on the imod-python documentation
* Set the channel priorities as mentioned above
* Then do the development install
* Install other packages as needed, with the channel priorities set, it should go alright
We're also aware the previous release is relatively old (October 2020), I think I'll make a new release next week.
Kristina Gumgowski, modified 3 Years ago.
RE: Error in imod.idf.open function from imod-python
Youngling Posts: 13 Join Date: 3/18/21 Recent Posts
Hi Huite,
I was about to post sth in this Forum how I solved the issue after all. In the end I went back to the older version of imod, because I could not make the development install work.
My solution was to create a new conda environment (as you suggested too) and install imod and all other packages from conda-forge. Curiously, I still got the geopandas error. I then followed the instructions from the link I mentioned in a previous post and de-installed the rtree package. After this was done I could run the code without getting an error on the packages imported. Since my input files changed as well in the meantime, the initial error why I opened this thread also did not show again. However, I still don't know why it appeared.
I will keep on working with the imod version from conda forge and update it regularly. I am not sure the mentioned "conda config" code will solve the issue here, because when I read the instruction on the imod-python webiste correctly the development install will still run over pip - or am I wrong? Or can a pip install still acces conda forge? I am really moving on thin terrain here.. I am still only starting to understand this part of python programming.
All the best
Kristina
I was about to post sth in this Forum how I solved the issue after all. In the end I went back to the older version of imod, because I could not make the development install work.
My solution was to create a new conda environment (as you suggested too) and install imod and all other packages from conda-forge. Curiously, I still got the geopandas error. I then followed the instructions from the link I mentioned in a previous post and de-installed the rtree package. After this was done I could run the code without getting an error on the packages imported. Since my input files changed as well in the meantime, the initial error why I opened this thread also did not show again. However, I still don't know why it appeared.
I will keep on working with the imod version from conda forge and update it regularly. I am not sure the mentioned "conda config" code will solve the issue here, because when I read the instruction on the imod-python webiste correctly the development install will still run over pip - or am I wrong? Or can a pip install still acces conda forge? I am really moving on thin terrain here.. I am still only starting to understand this part of python programming.
All the best
Kristina
Huite Bootsma, modified 3 Years ago.
RE: Error in imod.idf.open function from imod-python
Youngling Posts: 3 Join Date: 5/28/21 Recent Posts
Hi Kristina,
You got the original error because when we're opening an IDF, we look for a specific value to see if the data is single or double precision. If your IDF is somehow corrupted, the value might not come out right (we're looking for specifically a value of 1271 for single precision, 2295 or 2296 for double precision -- don't ask why). There was a typo in the iMOD manual originally, so I picked the wrong value initially when adding double precision support to imod-python. Current version of imod-python will accept both numbers. I think the GUI also accepts both (it does some aritmetic, and the same number comes out after rounding). If your IDF is in single precision, the error never appears.
If you installed imod-python via conda, or via the environment.yml, all the dependencies should already be there. So doing the development install via pip will only cause Python to import the local, newest version of imod-python. Pip should generally not have to install anything new (it'll say a lot of "requirement already satisfied").
Unfortunately, learning about package managers (like conda and pip) and dependencies are unavoidable when working with scripting languages and modern tools -- but "dependency hell" unfortunately applies to all programming languages, unless you want to write everything yourself. We are trying to make things easier with an upcoming release, see if we can package everything with a conda installer.
You got the original error because when we're opening an IDF, we look for a specific value to see if the data is single or double precision. If your IDF is somehow corrupted, the value might not come out right (we're looking for specifically a value of 1271 for single precision, 2295 or 2296 for double precision -- don't ask why). There was a typo in the iMOD manual originally, so I picked the wrong value initially when adding double precision support to imod-python. Current version of imod-python will accept both numbers. I think the GUI also accepts both (it does some aritmetic, and the same number comes out after rounding). If your IDF is in single precision, the error never appears.
If you installed imod-python via conda, or via the environment.yml, all the dependencies should already be there. So doing the development install via pip will only cause Python to import the local, newest version of imod-python. Pip should generally not have to install anything new (it'll say a lot of "requirement already satisfied").
Unfortunately, learning about package managers (like conda and pip) and dependencies are unavoidable when working with scripting languages and modern tools -- but "dependency hell" unfortunately applies to all programming languages, unless you want to write everything yourself. We are trying to make things easier with an upcoming release, see if we can package everything with a conda installer.