Publish Maps with Python

Publishing maps is one of the thing that we all love about the map making process. Having your maps reach a large crowd of people and shared across platforms can be one of the greatest achievement one can get when it comes to geospatial publishing.

Despite having fun and creating all the maps, it can become a very tedious job where you have to repeat the same process over and over again. Let’s take it this way:

You have been recently tasked to create and publish map layers for your Organization using ArcGIS Online in a scenario where the data should be published each and every day since the process is continuous.

How can we go through this process for the next 100+ days without getting bored or even end up being ignorant of some mistakes that we make along the way.

Python is here to rescue you

Did you know? – By creating code snippets for your Organization, you can save yourself sometime each and every day for that task? Python can be a really helpful tool especially when we can leverage the ArcGIS API for Python that we can get from the ArcGIS Developers platform.

Today, we are going to be publishing a simple map on the ArcGIS Online Organizational account from a CSV file. I have provided a checklist below for the few items that we are going to need for this tutorial.

Checklist

I have provided the link to the file for this tutorial which you can use incase you just want to explore and learn how it is done in a quick way. The file is for Districts in Zimbabwe.

If you have all the above listed items, you are good to go.

Just a quick layout of what we are going to be doing in this tutorial:

  1. Login to our account ArcGIS Online Account
  2. Use Pandas to read & visualize our CSV data
  3. Upload our CSV file
  4. Turn CSV file into a Feature Layer
  5. Finally, Add our new Feature Layer to a map

All these steps I have outlined are going to be carried out within this same notebook. Make sure to understand the syntax that is required for every operation that we are going to be performing within this tutorial.

Note: You can also use the ArcGIS Pro embedded Notebooks, the API is already pre-installed on installation of the software which automatically makes it easy to perform Notebook operations for Data Science

In case you weren’t here during the first tutorial, here is a resource which you can follow for installation of the ArcGIS API for Python within your local machine.

You can also watch it on YouTube

Introduction to ArcGIS API for Python

Importing Libraries

Just like what we mentioned in the Introductory guide, every operation within Jupyter Notebooks or that used Python Language as the basis for the operations requires some additional libraries to quicken the processes and automate these workflows.

Since we already downloaded the Python API, all we need to do is just call that into the Notebook and keep it ready for use within our space later down the road.

So right below, we are going to import the following Python libraries

  • GIS
  • Pandas and
  • getpass

getpass – The getpass() function prints a prompt then reads input from the user until they press return. The input is passed back as a string to the caller. Source.

So basically we need the getpass() function to be able to store my password for my ArcGIS Online account and send that as an authentication request. The reason why I called this function I don’t want anyone reading this to be able to see my password while I record the video tutorial or while I print this document. Its also a great feature for security purposes which keeps all sensitive information secure.

So let’s import these libraries into our Notebook.

from arcgis.gis import GIS
from getpass import getpass
import pandas as pd

Hint: Always remember to call all the librarries that you are going to be using right at the top. This should be the first thing you need to do before anything else

Authentication (Login In) to the ArcGIS Online Account

For us to get permission (be allowed) to interact with the platform, we need to be authenticated to perform the operations hence that requires that we are logged in.

So, I’ve created a password variable which is going to store the input from the getpass() function.

The variable gis is the one which is going to be used to authenticate the user based on the following parameters:

  • the site URL in this case which is the URL for the ArcGIS Online Platform
  • the username of the person and
  • the password of the user

All this is done via the GIS() method and the parameters to be passed are the one’s I have just mentioned above.

When you run the Cell, a small input box will appear below asking for your password

password = getpass()
gis = GIS("http://www.arcgis.com", "<your_username>", password)

Now that we have been authenticated, we need to move on to the next step.

Loading our Data (csv)

It’s time for us to use the Pandas library to view, read and import our data.

  • declared a variable in which you are going to store the path to the CSV file which you want to upload. csv
  • call the pandas read_csv function to read our data and
  • the head() function to print out the first rows within the data that we have specified as a parameter within the function. Remember: If the function is blank, head() function will print the first 6 rows as the default

Our data from the CSV file is then printed as output within our Notebooks.

csv = r"C:\Users\<Username>\Downloads\zwe_districts.csv"
districts = pd.read_csv(csv)
districts.head(20) 

Columns with no value in them will be displayed as NaN

The file path used above only works if you are using the file for this tutorial. Make sure you replace <username> with your computer username. And if you are using other folders, you need to place the correct path to those folders too.

We just want to see how many rows and how many columns are contained within our dataset. To do that we just call in shape after our variable and our information is displayed below in the format: (rows, columns)

districts.shape

Publishing An Item

We have seen what our data looks like and we are sure its the right dataset that we want to use. For some, if you would like to make changes to the data before publishing you could also do that again. (We will talk about how you can edit data using Pandas in another tutorial)

Now we need to publish an item to the ArcGIS Online platform based on our CSV file which is still within our local machine.

To do that, we need to declare a variable here which we just named item_csv which is going to store some property information for us. We are just going to put title as the only property for now.

We then have to call again another variable and this time we will name it csv_item in which we will call the gis library and give it a command that we need to add some content with these properties (item_csv) using this data (csv).

Remember that we already declared variables which contain the paths and data that we need to upload so we just call the variables instead of typing a huge piece of code which is just executed once

Lets call that variable csv_item to show us the output of our newly uploaded item within our ArcGIS Online platform.

item_csv = {'title': "ZWE Districts"}
csv_item = gis.content.add(item_properties=item_csv, data=csv)
csv_item

Create a Feature Layer

Since we have already uploaded our data as an item, it only takes two lines of code to create a Feature Layer from that item.

  • we create a variable which contains the name of our item and the publish function which has NO parameters at the moment
  • we just call our variable for us to verify that the item is now a Feature Layer

WE HAVE A FEATURE LAYER. GREAT!

Notice the Difference in MetaData between the Out[5] and Out[6] cell blocks

districts_item = csv_item.publish()
districts_item

In this step, we just want to verify that we really uploaded this feature layer to the ArcGIS Online platform. Let’s just run a basic search query.

In case you are lost – refer to the previous tutorial in the video at the beginning to understand more about searching with the Python API

OK. So our Feature Layer has been published we just confirmed it. If yours didn’t publish, try checking that all the cells above have been ran without any errors and check especially for any typos that you might have made.

districts_lyr = gis.content.search(query="Districts*", item_type="Feature Layer")[0]
districts_lyr

Add Layer to Map

We’re getting ready to add our newly added layer to the map. Before we do that I am just going to show you one important thing that has to do with Feature Layers.

All feature layers within the ArcGIS Platform have an Index. This is just the number contained at the end of the URL for the whole feature layer. This Index helps us indentify the dataset which we want to use.

Imagine if we had publised a Geodatabase File with multiple layers all combined and attached to one URL. We would need to know which is which because we could have line data and point data within the Geodatabase file and they have the same URL. The only thing to separate them is the Index at the end of the URL after the trailing slash (/).

These Indexes start at Zero(0). So if we have just one feature within a layer, that automatically means that its index is Zero. So I am just going to use that here and so that you know how Layer Indexes work.

districts_lyr = districts_lyr.layers[0]

Call the Map with Location

We have called a map for use to have a basemap in order to easily visualize the data which we are about to add.

Remember from our last tutorial, we discussed on how we can add a basemap to these maps. In my case, I am just going to leave that as default.

map = gis.map("Zimbabwe", 6)
map

Finally, our last step is to load the layer onto the map by simply calling the add_layer() function like what we did previous in the last tutorial again.

map.add_layer(districts_lyr)

The variable in which we selected an index for the layer is the one which is going to be our parameters for this function.

Wait a few seconds, depending on your network connection and your Feature Layer will be plotted onto your map.

Zimbabwean map of Districts mapped from our CSV file

You should see the map like the one above if you followed through the correct steps of this tutorial.

If you didn’t face any errors along the way and managed to get the data onto the map then;

CONGRATULATIONS

Congratulations

You’ve managed to complete your first step in #GISAutomation

If you were using Visual Studio Code for this tutorial, then your file should look like below:

from arcgis.gis import GIS
from getpass import getpass
import pandas as pd

password = getpass()
# Make sure you replace <username> with your ArcGIS Online Developers Account username
gis = GIS("http://www.arcgis.com", "<username>", password)

# Change the path to suit your location
csv = r"C:\Users\<username>\Downloads\zwe_districts.csv"
districts = pd.read_csv(csv)
districts.head(20) 

# The CSV file structure
districts.shape

# Publishing an Item
item_csv = {'title': "ZWE Districts"}
csv_item = gis.content.add(item_properties=item_csv, data=csv)
csv_item

# Create a feature layer
districts_item = csv_item.publish()
districts_item

# Verifying to see if our Layer was actually published
districts_lyr = gis.content.search(query="Districts*", item_type="Feature Layer")[0]
districts_lyr

# Indexing out Layer
districts_lyr = districts_lyr.layers[0]

# Display a map
map = gis.map("Zimbabwe", 6)
map

# Add our layer to the map
map.add_layer(districts_lyr)