The GIS Jester told a Story to make the King of GIS laugh

In July I attended the Esri User Conference in San Diego.  I had not been to the UC since 2011.

Back in 2011, I was invited to the Special Achievement in GIS (SAG) awards ceremony at the UC. The list of award winners and the breadth and depth of their work with GIS was amazing. It was a privilege to mingle with the achievers and to hear the late Roger Tomlinson, The “Father of GIS”, speak to hundreds of his achiever children.  I was sort of a GIS under-achiever but glad to be there.

Roger acknowledged the Father of GIS appellation then called Jack Dangermond the “King of GIS”.

It was memorable.  I got to shake The GIS Father’s hand and stand with The GIS King for a photo op.  Not for any SAG achievement.  They gave me the THIRD PLACE award for storytelling.  This was back before #storymaps were a thing.   So it was easy to win.

For three short seconds, I gripped the GIS King’s hand while the photographer adjusted our pose.  In that moment, I told the King that 20 years before, I had cast out bad spirits and given up religion and converted to Esri and GIS.  A true believer.  And now he is my inspiration to do good in the world. The King laughed, the camera clicked, and the staff waved me off to make room for the next photo op.

2011-07 Jack Dangermond and Don cropped 700x

Jack Dangermond presents the THIRD PLACE award for Story Telling with Web Maps

Metadata Lite: Publishing metadata cheap (or not at all) through ArcGIS Server

You can use ArcGIS Desktop to author metadata for data layers in your geodatabase, and to publish your data as web services on ArcGIS Server.  But you can’t publish your geospatial metadata with those web services.

Esri will say, Not true.  You can publish your metadata to the Web through Esri’s free Geoportal Server.  Then point your metadata on the Geoportal Server to your map services on your ArcGIS Server, like this:
Right.  But that means you have to do the same task twice — send your metadata to your geodatabase and send it again to your geoportal.  Besides maintaining two applications servers for this purpose.
This redundancy sits atop the already cumbersome and expensive process of authoring, publishing, and maintaining FGDC/ISO-compliant geospatial metadata.
Are there less expensive options?  Even Esri’s FGDC/ISO-compliant metadata guru seems to think so.  Here is a slide from Marten Hogeweg’s geoportal workshop at the 2011 Dev Summit:
(The title/question is mine, not Marten’s.)

Marten was talking about different metadata for different audiences.  The public does not want or need FGDC/ISO compliant metadata.  So, how can we meet the public demand, as well as the needs of our professional partners in the GIS community?  I suggest a middle path – Metadata Lite – that’s already suggested in Marten’s diagram.
Yes, “Verbose Metadata is Desired” for the GIS Specialist Community.  But today more GIS professionals are concerned with pushing data out to the public.  Knowing that, it seems to me that Esri (unconsciously?) moved away from fully-compliant metadata  when they “forgot” to support metadata publishing through ArcGIS Server 9.3 and 10.  And now we see mere “tagging” promoted in Marten’s metadata/geoportal presentations.    
Take a look at ArcGIS.com.  At the Esri MUG last week, Clint Brown told us that millions of maps are added to ArcGIS.com each month.  But where is the geospatial metadata that enables search in a geoportal?  There is none at ArcGIS.com.  You can’t search through others’ FGDC/ISO-compliant metadata, and you can’t publish your own there.  
There is only Metadata Lite — tagging to give a minor boost to the ArcGIS.com search function:

If you want to publish descriptive info about your map data, there are a couple of free/low-cost options.  First, you can publish your data in map apps at ArcGIS.com.  (See my other posts about using ArcGIS.com for your organization’s geodata portal.)  Then simply author the descriptive information in the details page.  San Mateo County GIS has done it this way, using a text template to ensure that all the legal info is included, like in this example:

Even better, I think, is to publish your geospatial metadata through your ArcGIS Server REST catalog.  When it’s in the REST catalog, it doesn’t intrude on the public attention span, yet GIS professionals can find it when they need it.

This is what Metadata Lite looks like from San Mateo County, published through ArcGIS Server to the county’s REST catalog:

But 90 percent of the time, if you open up a REST service page like this, all the fields are blank, except a few that are machine-generated,  like Extent and Spatial Reference.  Why can’t we simply author metadata that shows up in the REST catalog?  Because …

You need a wiring diagram.  Because some of the REST catalog fields are populated from the ArcMap Document Properties tab.  Others from the ArcMap Data Frame Properties tab.  Others from the ArcGIS Server Manager Properties tab.  Like this:
Besides that, the field names are different for the same information found in the source (e.g. ArcMap) and the destination (REST page).  Like this:
So, here are a couple of wiring diagrams to help you populate the REST catalog fields:
2.  Metadata Lite live map service – hosted at University of Maryland, shows where the REST catalog fields values originated.
(From a presentation I gave at the 2011 Esri MUG)

Esri URL Parameters (Part 2: Webmap, Mapkey, Open, or Itemid… It’s all the Same)

My last post showed how San Mateo County (Calif.) used URL parameters to create a Public Maps Gallery with options for different map viewers, like in this example.   We authored our maps at ArcGIS.com to show map services on the County GIS server.  Then we sent the unique map ID as a URL parameter to four different map viewers.
But there’s a catch with this approach:   Esri uses a different URL parameter with each map viewer – to send the same information.  (It seems like the Esri dev teams for each of the map viewer APIs were not talking to each other.)  I couldn’t find a single listing for all the variations, so I put together this list myself:
For this Esri Map Viewer…
…This is the URL parameter for the ArcGIS.com web map
ArcGIS.com Map Viewer
?webmap=
Javascript Templates hosted at  ArcGIS.com
?webmap=
Esri Javascript Templates hosted on a local web server
?mapkey=
ArcGIS Explorer (Silverlight) Map Viewer      
?open=
ArcGIS Viewer for Flex
?itemid=

Configurators Love Esri’s URL Parameters (Part 1: One Parameter, Lots of Map Apps)

URL parameters are no big deal for web map programmers.  They use URL parameters all the time to configure complex web applications at use-time.   But they’re a big deal for Configurators  —  map makers who are only programmer wannabes.  We could never make a living programming.  But we can do a lot of web mapping by reconfiguring other people’s code.

Esri has opened up lots of possibilities for Configurators.  It’s possible for Configurators to adapt the Esri Javascript API code samples, but the learning curve is steep.   Their Flex Viewer is easier to configure.  Just change XML tag content.  Easiest of all are the web map apps at ArcGIS.com.  Once the map is authored, the unique map ID can be sent to several pre-built viewers.

San Mateo County (Calif.) took advantage of web map parameter to create a Public Maps Gallery that gives users several viewing options.  We authored our web maps at ArcGIS.com to show map services on the County GIS server.  Then we sent the web map unique ID as a URL parameter to several different map viewers.

The map of Whitehouse Creek – Parcels Adjacent to Rare Species Habitat is a good example.

Additional map viewer options appear at the bottom of this simple viewer:

  • Full-Service Map Viewer (ArcGIS.com) is the standard Javascript map viewer hosted at ArcGIS.com.
  • Presentation Map is ArcGIS Explorer Online map viewer at ArcGIS.com, built with Microsoft Silverlight.  This map viewer requires the Silverlight plug-in.  It has many advanced features for configuring the map layers and popups, and for creating online presentations for “telling the story with maps.”
  • SMC Flex Map is Esri’s ArcGIS Server Map Viewer for Flex, which has been locally configured and hosted on the San Mateo County web server.
  • SmartPhone Map is a simple Javascript map viewer template hosted at ArcGIS.com.  We could have customized and hosted this on the San Mateo Countyweb server instead.  Its simple format is useful for small-format mobile devices.  But this template has no capability to turn map layers on/off, so it’s limited to showing only those map layers the map author made visible when the web map was saved.

There are some gotchas with the URL parameter for the unique map ID.  You can see this by comparing the map ID parameter that is actually sent in the URL for each of the map viewers.  More about this in my next post.

Why doesn’t Esri support metadata publishing through ArcGIS Server and ArcGIS Online?

Part of my task at San Mateo County GIS (Apr-Oct 2011) was to set up a process for publishing geospatial metadata along with the County’s map services.  We looked at Esri’s free Geoportal Server.  But we decided that was overkill for the County’s data sets.  And we were also having trouble with metadata authoring and publishing with ArcGIS v10.  We needed to publish metadata through ArcGIS Server and the REST API, or through ArcGIS Online where the public would access our map services.  But we couldn’t do either one.
Why doesn’t Esri support metadata publishing through ArcGIS Server and ArcGIS Online?
This is what I’ve concluded from my conversations with several Esri customer reps and product managers, including  Sud Menon:
Back in the late 90s the federal government wanted to save money by sharing data so it didn’t have to pay to collect the same data twice.  The Federal Geographic  Data Committee (FGDC) thought it was a good idea to develop a metadata standard and taxonomy to enable fast searching on tags, keywords, and technical data descriptions.   FGDC figured that vendors like Esri should support this with geosptial metadata creators and search capability.  Esri agreed, probably because the USG is their biggest customer. 
   
So Esri built the first geoportal that provided the search and share capability for Dept of Interior’s geodata.gov site, based on FGDC standard metadata.  Esri (and everyone else in big government) thought that government organizations could mandate that their GIS units would make the effort to document all their geodata in the FGDC metadata standard.   But it didn’t happen, because it’s too big an investment for most organizations.
When Esri developed ArcGIS Online, they opted not to support full FGDC metadata either.  And when they developed ArcGIS Server v9 and v10, they again gave it little attention.  Esri focused instead on simple key words and minimal tag information — less metadata for more audience – to keep the hurdle low for sharing geodata. 
Esri says that ArcGIS Server will support metadata publishing through the geodatabase in the future.   In Q&ADiscussion before the 2011 UC, Esri announced:
“ArcGIS Server 10.1 will automatically capture and store basic metadata about the GIS services you create and allow you to enhance metadata documentation with descriptions, summaries, tags and other information. Any client accessing these services as well as anyone using ArcGIS.com and the Portal for ArcGIS will be able to leverage this information….  you can create and update your GIS service metadata using the tools that are built-in to ArcGIS. This metadata will be available via a simple URL.”
Lots of organizations went to the trouble of creating structured metadata.  But San Mateo County has only a loosely-structured html-based metadata catalog.  The best approach for the County now seems to be a blend of FGDC-like metadata documentation for geodata that is complex, that must be accurate and precise, and/or is a component of a geodata model that is shared among GIS experts.  And to author and publish a more compact set of simpler (but still standardized) tags and keywords for most geodata shared with partners and the public.
Standard tags is what we used for “metadata lite” with the San Mateo County Geodata Catalog.

AGXO Ltd.

Esri’s Bern Szukalski visited the City of Philadelphia GIS team in early December to show and discuss ArcGIS Explorer (AGX) capabilities. We were considering using AGX or AGX Online (AGXO) to present the city’s GIS strategic plan to stakeholders. Bern asked if we thought AGXO would be adequate for this project and its objectives, as outlined by the city’s Director of Enterprise GIS. I had no reason to doubt it at the time. Now I’m not so sure.

AGX Online is limited to basic labels and popups. It will fall short on a couple of the projects objectives: It won’t present the GIS Strategic Plan with a lot of “impact” to city government audiences, nor display “Powerpoint-like” design details, as the GIS Director hoped.

Working the AGXO Frederick Douglass presentation ( http://bit.ly/fd-birthplace ) showed me that AGXO is limited to:

  • Basemaps pan, zoom, and switch
  • Titles and text labels
  • Point, line, and polygon graphics
  • Popups from the graphics that are limited to
    • title and short text
    • displayed image or video
    • one hyperlink

That’s all there is.

The strategic plan presentation might be improved somewhat with enhancements in the next release of AGXO, which Bern demo’d at Esri’s Federal UC in January. He also described it at http://bit.ly/f4pcqT . Enhancements are:

  • map feature templates
  • better feature and annotation editing wizards
  • better user experience — streamlined and simplified for ease of use.

The first two will make authoring more efficient but won’t expand AGXO’s presentation quality very much. Better user experience will help. This is something that many of my Frederick Douglass AGXO beta testers critiqued.

Maybe Esri can give me access to the new release in time for use on this project.

Can’t live without (someone else’s) ArcGIS Server

I’ve paid close attention to Esri’s promotion of their “GIS for everyone” resources at ArcGIS.com.  Just go there, Esri says, and create your own maps by “mashing up” satellite imagery, street or topographic basemaps, census data, environmental data.  And add your own lines, polygons, markers, and labels.

That works if you can don’t mind using everyone else’s data but your own.

So far, if you want to create maps on ArcGIS.com with your own data, you have to own ArcGIS Server.  Or borrow one.  Then publish your data as map services on ArcGIS Server.

You can’t upload your own data to ArcGIS.com from Excel, or as Esri shapefiles, or as Esri layer packages, and add them to your “Map for Everyone” at ArcGIS.com.  You can post those data files,  and others can download them to display in desktop map viewers.  But that’s not what we’re talking about here.

Thanks to Dr. Jack Ma at Univ. of Maryland, I still have access to their ArcGIS Server after graduating from the MPS GIS program.  (Jack gets a little payback – and occasional class presentation from me.)  Here’s the REST directory:
http://129.2.24.163/ServerMPSArcGIS/rest/services

Two ArcGIS Servers are better than one.  Especially when Server #1 has limited tech support.

While working the past six months for the City of Philadelphia GIS Director, I also connected with Maurie Kelly, director of the PASDA geodata portal at Penn State.  I’m grateful to Maurie for providing map service publishing support for Choptank River Heritage geospatial data.  (Maurie’s generous logic is:  Choptank flows into Chesapeake … Susquehanna flows thru PA into Chesapeake … So, yeah, let’s host it on PASDA.)  Here’s the REST directory:
http://146.186.163.133/arcgis/rest/services/ChoptankRiverHeritage/MapServer/

Once you have map services published on ArcGIS Server, wow, there’s so much you can do:  ArcGIS.com mashups, AGXO map presentations, Flex map viewer from my own web site, data downloads, geoportal, geoprocessing services.  Even output KML for Google-based services.