February 7, 2025

Data SGP – Visualizing and Analyzing ARM Data

The Southern Great Plains (SGP) atmospheric observatory gathers data about cloud, aerosol, and atmospheric processes, helping scientists understand and model the Earth’s climate. The data from SGP are used in models that improve global weather forecasting, and help scientists better understand the interactions of clouds and aerosols with other parts of the atmosphere and our changing planet.

The SGP Observatories are the primary field measurement sites in the ARM network. These observatories measure a wide range of environmental variables including cloud properties, temperature, humidity, wind speed and direction, air pressure, solar radiation, and water vapor. The data gathered at these observatories are made freely available through ARM’s Data Discovery website.

ARM data are critical to scientific studies that advance understanding of the physics of the atmosphere and improve climate models. The ARM research community has developed a variety of innovative methods for analyzing the enormous volume of ARM data. These techniques include data visualization, data integration and aggregation, machine learning, and high-performance computing. The data sgp project builds on these advances to provide an efficient, user-friendly way for scientists to visualize and analyze the large amount of data gathered by ARM and made available through Data Discovery.

In education, a student growth percentile (SGP) is a number that indicates how much a student has grown in relation to his or her academic peers over time. Teachers can use SGP information to determine whether students grew more than, less than, or as expected. The SGP system allows teachers to identify student growth in several formats: Window Specific SGP, which is reported for a specific window of testing; and Current SGP, which is a real-time estimate of a student’s SGP for the most recent window of testing.

A student’s SGP is based on five years of vertically scaled assessment data. The first column of the data set sgpData provides a unique student identifier, while the remaining columns provide assessments scores in each year. The SGP analysis functions, studentGrowthPercentiles and studentGrowthProjections, require data in LONG format and expect the embedded SGPstateData meta-data to be present.

Using the sgpData data set and higher level SGP package functions to conduct SGP analyses is straightforward. Any issues that arise typically revert to data preparation issues. The sgpData vignette provides detailed information on how to manage this type of data.