Using importlayers to import Data into GeoNodeΒΆ

The geonode.layers app includes 2 management commands that you can use to load or configure data in your GeoNode. Both of these are invoked by using the manage.py script. This section will walk you through how to use the importlayers management command and the subsequent section will lead you through the process of using updatelayers.

The first thing to do is to use the –help option to the importlayers command to investigate the options to this management command. You can display this help by executing the following command:

$ geonode importlayers --help

This will produce output that looks like the following:

Usage: manage.py importlayers [options] path [path...]

Brings a data file or a directory full of data files into aGeoNode site.  Layers are added to the Django database, theGeoServer configuration, and the GeoNetwork metadata index.

Options:
  -v VERBOSITY, --verbosity=VERBOSITY
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings=SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath=PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Print traceback on exception
  -u USER, --user=USER  Name of the user account which should own the imported
                        layers
  -i, --ignore-errors   Stop after any errors are encountered.
  -o, --overwrite       Overwrite existing layers if discovered (defaults
                        False)
  -k KEYWORDS, --keywords=KEYWORDS
                        The default keywords for the imported layer(s). Will
                        be the same for all imported layers if multiple
                        imports are done in one command
  --version             show program's version number and exit
  -h, --help            show this help message and exit

While the description of most of the options should be self explanatory, its worth reviewing some of the key options in a bit more detail.

  • The -i option will force the command to stop when it first encounters an error. Without this option specified, the process will skip over errors that have layers and continue loading the other layers.
  • The -o option specifies that layers with the same name as the base name will be loaded and overwrite the existing layer.
  • The -u option specifies which will be the user that owns the imported layers. The same user will be the point of contact and the metadata author as well for that layer
  • The -k option is used to add keywords for all of the layers imported.

The import layers management command is invoked by specifying options as described above and specifying the path to a single layer file or to a directory that contains multiple files. For purposes of this exercise, lets use the default set of testing layers that ship with geonode. You can replace this path with the directory to your own shapefiles:

$ geonode importlayers -v 3 /var/lib/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/

This command will produce the following output to your terminal:

Verifying that GeoNode is running ...
Found 8 potential layers.
No handlers could be found for logger "pycsw"
[created] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_administrative.shp' (1/8)
[created] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_coastline.shp' (2/8)
[created] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_highway.shp' (3/8)
[created] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_location.shp' (4/8)
[created] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_natural.shp' (5/8)
[created] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_poi.shp' (6/8)
[created] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_water.shp' (7/8)
[created] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/single_point.shp' (8/8)

Detailed report of failures:


Finished processing 8 layers in 30.0 seconds.

8 Created layers
0 Updated layers
0 Skipped layers
0 Failed layers
3.750000 seconds per layer

If you encounter errors while running this command, you can use the -v option to increase the verbosity of the output so you can debug the problem. The verbosity level can be set from 0-3 with 0 being the default. An example of what the output looks like when an error is encountered and the verbosity is set to 3 is shown below:

Verifying that GeoNode is running ...
Found 8 potential layers.
[failed] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_administrative.shp' (1/8)
[failed] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_coastline.shp' (2/8)
[failed] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_highway.shp' (3/8)
[failed] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_location.shp' (4/8)
[failed] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_natural.shp' (5/8)
[failed] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_poi.shp' (6/8)
[failed] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_water.shp' (7/8)
[failed] Layer for '/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/single_point.shp' (8/8)

Detailed report of failures:


/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/gisdata/data/good/vector/san_andres_y_providencia_administrative.shp
================
Traceback (most recent call last):
  File "/Users/jjohnson/projects/geonode/geonode/layers/utils.py", line 682, in upload
    keywords=keywords,
  File "/Users/jjohnson/projects/geonode/geonode/layers/utils.py", line 602, in file_upload
    keywords=keywords, title=title)
  File "/Users/jjohnson/projects/geonode/geonode/layers/utils.py", line 305, in save
    store = cat.get_store(name)
  File "/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/geoserver/catalog.py", line 176, in get_store
    for ws in self.get_workspaces():
  File "/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/geoserver/catalog.py", line 489, in get_workspaces
    description = self.get_xml("%s/workspaces.xml" % self.service_url)
  File "/Users/jjohnson/.venvs/geonode/lib/python2.7/site-packages/geoserver/catalog.py", line 136, in get_xml
    response, content = self.http.request(rest_url)
  File "/Library/Python/2.7/site-packages/httplib2/__init__.py", line 1445, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/Library/Python/2.7/site-packages/httplib2/__init__.py", line 1197, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/Library/Python/2.7/site-packages/httplib2/__init__.py", line 1133, in _conn_request
    conn.connect()
  File "/Library/Python/2.7/site-packages/httplib2/__init__.py", line 799, in connect
    raise socket.error, msg
error: [Errno 61] Connection refused

Note

This last section of output will be repeated for all layers, and only the first one is show above.

This error indicates that GeoNode was unable to connect to GeoServer to load the layers. To solve this, you should make sure GeoServer is running and re-run the command.

If you encounter errors with this command that you cannot solve, you should bring them up on the geonode users mailing list.

You should now have the knowledge necessary to import layers into your GeoNode project from a directory on the servers filesystem and can use this to load many layers into your GeoNode at once.

Note

If you do not use the -u command option, the ownership of the imported layers will be assigned to the primary superuser in your system. You can use GeoNodes Django Admin interface to modify this after the fact if you want them to be owned by another user.