Data Gathering

Goals and Objectives:

The goal of this lab is to learn how to find and download data from a variety of sources online, manipulate and join it using ArcGIS, project it into one coordinate system, and build and design a geodatabase to store the data.  It is also intended to provide more exposure to python coding in order to automate geoprocessing of data.

Methods:

The basic workflow of downloading each data set for this lab began by downloading zip files into a temporary directory so the large files can be easily deleted later.  They were then extracted into a working folder.  The data sets were projected and clipped/extracted and then loaded into a geodatabase.

The process began by finding and downloading several data sets.  The first one was the Polyline Railway Network file from the US Department of Transportation.  Next, the land cover for the state of Wisconsin (NLCD 2011 Land Cover) was downloaded via the USGS National Map Viewer.  The same website was used to download the elevation data sets (1/3 arc-second DEM).  This required downloading both n44w092_13 and n45w092_13 in order to get data for all of Trempealeau County. From the USDA Geospatial Data Gateway, the Wisconsin Cropland Land Cover was downloaded. From the Trempealeau County Land Records division website, the entire Trempealeau County file geodatabase was downloaded.  Finally, from the USDA NRCS Web Soil Survey website, the Trempealeau County soils data was downloaded.

The soils data was downloaded as a Microsoft Access geodatabase.  In order to be able to properly use this data, it was manipulated with Microsoft Access by importing the .txt files into the geodatabase schema.  This connected the actual data to the geodatabase template file.  In ArcCatalog, the soils shapefile was then imported into the TMP geodatabase. The component table was also imported into the TMP geodatabase. A relationship class was created to join the component information to the new soils feature class. Both of these were added to ArcMap and joined based on the relationship class.  The NTAD rail lines shapefile was added to the map, clipped to the Trempealeau County boundary, and added to the TMP geodatabase.  This projected the data into the proper coordinate system. The DEMs were then added to the map and combined using the Mosaic to New Raster tool.

The 3 raster datasets that were downloaded, along with the TMP geodatabase were moved to their own separate folder.  Then Pyscripter was used to create a python script that projected the rasters, extracted them to the TMP county boundary, and loaded the .tifs into the geodatabase. The code that was created to do this can be seen in the Python Scripts post below.  The results of this script can be seen in Figure 1 below.

Figure 1: The resulting layers that were created from using Python to project and extract the 3 raster datasets.
Legends for Land Cover and Crop Cover were omitted intentionally due to lack of space available. Information about the significance of these values can be found at the source websites linked above. 

Data Accuracy:

Figure 2:  Analysis of the metadata for each dataset downloaded for this lab.

Conclusions:

In general, the data seemed to come from reputable sources.  It will be important to keep in mind the scale when conducting further analyses in order to preserve data integrity.  Many of the datasets only provided a few of the data quality parameters so the others were estimated based on what was given. This could be a potential concern relating to the datasets used.  It was especially a concern when the scale was estimated based on the resolution values. While the sources seemed reputable, the often incomplete metadata leaves a level of uncertainty when it comes to the quality of the data. It was especially concerning when metadata did not include information about the various different accuracy levels, or the accuracy levels were left to be inferred by other information.  The USDA soils dataset seemed to have the most complete metadata available.  It was difficult to evaluate the TMP geodatabase because there was very little metadata for the geodatabase as a whole.  The majority of information available is for individual feature classes and many of these feature classes have very different metadata associated with them.  Overall, it would be a lot better if the metadata were to explicitly state these data quality measures instead of relying on users inferring and estimating them.  This would improve confidence levels greatly.