2016 Open NY Annual Report

Open Data puts data into the hands of all and gives it context - making it easily consumable, understandable, and usable. We strive to achieve our mission of “Quality by Design” by providing detailed information about our data and demonstrating our commitment to outstanding value and quality.

Through Open NY, the State is advancing a data-driven strategy that promotes transparency and accountability, supports the sharing of information, and fosters an environment for innovation, economic development, and scientific research. We are grateful to our partners across the State and look forward to continuing this vital work in the year to come.

Extraordinary growth highlights Governor Cuomo’s unwavering commitment to provide public access to government information, deeply rooted in the Governor’s agenda for New York State. The breadth and depth of available data is unprecedented. Open NY provides access to a wide range of data to analyze and trend - from datasets with hundreds of records to datasets with millions of records, from snapshots of data to decades of data. The number of data catalog items has increased more than fivefold since launch, and the number of records has more than tripled in the last two years. A global platform, access to Open NY has increased nearly threefold with visitors from over 12,000 cities worldwide, including over 8,100 U.S. cities, across 200 countries and territories, and all 50 States and Washington D.C. All categories have experienced double-digit growth, with many categories more than doubling and tripling the number of data catalog items available.

Over 1,375 data catalog items containing over 160 million records are currently accessible for reuse across 64 government entities and ten categories. Moreover, all Open NY tabular data resources can be accessed through a consistent, well-documented Application Programming Interface (API) that provides software developers with a simple, familiar, standards-based mechanism for integrating Open NY data into their software applications. Highlights among the numerous datasets added in the last year include:

• Over 20 datasets of detailed financial information on public authorities from the Authority Budget Office (ABO), including financial data, salaries, procurement reports, debt schedules, real property transactions and various economic development projects reported by public authorities to the ABO.
• New hourly and directional traffic flow data, with over 20 million rows, is available to explore, as is the latest full week 15-minute directional traffic flow snapshot from the NYS Thruway Authority.
• NYSERDA’s Residential Baseline Study data, from a first-of-its-kind study that compiled energy-related information from more than 3,000 homes in New York, is now available to the public on for use in identifying opportunities for energy efficiency improvements.
• Active Real Estate Salespersons and Brokers licensing data from New York State Department of State (DOS) is available to browse, filter, sort, and export.
• State Business Filings. This dataset includes over two decades of monthly filing statistics categorized by entity type and document type for all entities that file formation, authorization and associated documents with the Department of State’s Division of Corporations. Such entities include, but are not limited to: Limited Liability Companies, Business Corporations, Cooperative Corporations, Limited Partnerships, and Not-for-Profit Corporations, and more.
• Managed Care Consumer Guide data provides information about the quality of care for different health plans. Consumers can use this data to choose a managed care plan that meets their health care needs and the needs of their families.
• Registered Cooling Tower location data shares reported information about the type of equipment; inspection dates; inspection and testing results; and other regulation requirements that protect public health.
• Birth and Death Vital Statistics data contains demographic and population information available on New York State pregnancies, deaths, and marriages by age group, gender, race/ethnicity, and residence county that can be trended over time.

In addition, New York State continually updates existing information available on Open NY. Below are some highlights of interesting and popular data:

• MTA turnstile usage data 2014, 2015, 2016. Millions of rows can be explored where usage is broken down to daily and hourly periods for entry and exits to/from stations.
• Food Service Establishment Inspections. Get the latest food service establishment inspection results for restaurants, schools, summer camps, temporary food establishments, and more, filtered by facility, city, and county.
• Ancestry searches are facilitated by the Vital Records Office within the Department of Health. The Genealogical Research Death Index contains a variety of genealogy criteria on decedents, date of death, gender, age at death, and NYS file number that will allow the public to search for individuals who died at least 50 years ago in New York State, outside of New York City. To quickly research the Genealogical Research Death Index data starting in 1957, use this interactive search and filter version.
• Department of State Active Business Corporations since 1800. This dataset contains information on all active corporations as of the last business day of the specified month and year.
• Farmers’ Markets in New York State have again reached an all-time high, now with over 800 markets located across the state open either year round, or by season. View the continually updated, current locations of farmers’ markets that offer consumers plenty of New York-made items, including vegetables, fruits, meat and dairy products, honeys, maple syrups, baked goods, wines and more.
• Solar Power. New York’s role as a leader in shaping the future of energy remains firmly in place. Data is available for over 40,000 solar projects that have participated in NYS Energy Research and Development Authority’s open enrollment solar incentive programs, including the large ramp-up of solar under NY-Sun. View cost and energy savings by zip code, county, or in the aggregate across the state. A map of installation locations is also included.
• Wineries, breweries, cideries, and distilleries. New York is today home to over 800 beer, wine, spirits, and cider manufacturers. Under Governor Cuomo’s leadership, New York has partnered with the beverage industry to create new farm-based manufacturing licenses, launch aggressive marketing and promotional initiatives, modernize the law, ease restrictive regulations, and reduce the time it takes to open a business. As a result of these efforts, the number of New York craft beverage manufacturers has more than doubled since 2011.

Empowering the Public with Data

Open Budget: Openbudget.ny.gov

Open Budget provides single stop access to financial data including details on budgets, revenues, appropriations, school aid and actual spending. Open Budget makes data available in a format that can be searched, explored, analyzed, and downloaded. In addition, a budget exploration tool makes it easier for visitors to more easily understand the state budget through an interactive, graphical presentation with multiple options for navigation. Use the lookup tool to view and download disbursement information for the current budget year budget and actuals and prior years going back to fiscal year 1995 for all governmental funds. Data can be filtered by function, agency, fund type, or financial plan category, and results can be exported for further review and analysis.

Open Data: data.ny.gov

New York State’s Open Data portal has dramatically increased public access to one of the State’s most valuable assets: data. The site provides unprecedented, user-friendly access to freely available machine-readable data promoting re-use, analysis, and innovation. The public can now view catalogs of data sets and/or download datasets in multiple formats that enable greater utilization and maximize the data’s derivative value.

Health Data NY: health.data.ny.gov

The site promotes consumer education through visualizations of data utilizing maps and graphs and supports health research through publishing both aggregate and granular-level data, to improve healthcare for all New Yorkers. Data available via website is also simultaneously available at the main Open NY website.

Official New York State Website: NY.Gov

One year after its launch, the newly redesigned New York State website has experienced record growth. The new design of NY.Gov put New Yorkers at the center and made it easier than ever to utilize and learn more about government services. The website improved customer service, design, navigation, search, and accessibility from virtually any device. The website’s overhaul was the first major redesign in 15 years, and has set a new bar for digital government services through its user-centric approach.

A key part of the NY.Gov user experience is enhanced integration with Open NY, including a new section that highlights relevant content customized for the user’s location. In addition, the State has extended its vibrant, new design to Open NY, providing a seamless, consistent experience, and making it easier for the public to discover the vast amount of freely accessible and reusable data. NY.gov puts information and resources at the fingertips of New Yorkers as never before, reflecting the Governor’s commitment to government service and efficiency.

OASAS Treatment: Statewide Bed Availability Dashboard Application

The new web-based service helps connect New Yorkers struggling with addiction to treatment. The New York State Office of Alcoholism and Substance Abuse Services' (OASAS) bed availability dashboard collects bed availability from State-certified alcohol and substance use disorder treatment providers daily and makes it available in real-time on the agency's website.

Atlantic Ocean and Climate Change & Resilience: DOS Geographic Information Gateway

The Department of State’s Office of Planning & Development’s Ocean and Great Lakes and Climate and Resilience program launched a website dedicated to coastal and environmental data. The Geographic Information Gateway is an educational and user-friendly website that identifies New York’s diverse land and offshore assets so residents, local governments, educators, businesses and others can use this information when making planning and development decisions. The website provides public access to free and reliable geographic data, real-time information, interactive tools, and expert knowledge on New York’s resources, including climate change and community resilience activities.

Health Care by the Numbers Curriculum

An Innovation grant awarded to the New York University School of Medicine (NYU SOM) from the American Medical Association’s Accelerating Change in Medical Education Program, is teaching students how to use “big data” from Health Data NY to improve patient care coordination and quality of care. Known as the NYU ACE Program, a key foundational aspect of the “Health Care by the Numbers Curriculum” is learning to use authentic, free, and publicly available clinical data and open data resources.

A primary open data resource is the NYS DOH Statewide Planning and Research Cooperative System De-identified data (SPARCS), which is available on Health Data NY. Data extracts are downloadable for projects and research studies. According to Marc Triola, MD, Associate Dean for Educational Informatics, this project is designed to “…prepare our graduates to meet the future needs of the ever evolving healthcare delivery system by giving them the tools and skills necessary for not just the individual patient, but an entire population of patients." The Clinical Data Student Project created an interactive website for the 20 most common inpatient hospital admissions. For example, the variations in type of procedure and most common procedures can be quickly searched and compared.

Visualizations can render complex data into images which maximize the understanding of data by telling compelling stories. Visualizations also allow for the direct interaction with data in ways that are dynamic and engaging. They can identify trends, help distill increasing amounts of information, and transform raw data into actionable insight. The following examples demonstrate the power of depicting Open NY tabular data in a visual and interactive way.

NYS Thruway Traffic Flow

The interactive visualization below, created by the Open NY team for demonstration purposes, displays how open data might be presented to a broad audience. It enhances the understanding of travel patterns and use of various sections of the New York State Thruway by showing passenger vehicle flow between every interchange on the ticketed thruway system. The team used two distinct NYS Thruway Authority datasets, Thruway Interchanges and Exits and Origin and Destination Points for All Vehicles – One Week 15 minute Snapshot. Standardizing data enhances the capacity to combine and synthesize related datasets (nearly 725,000 rows), maximizing analytics and the ability to discern patterns and trends.

NYS Department of Transportation (DOT) – Average Annual Daily Traffic (AADT)

The interactive visualizations below were created using the NYS DOT Average Annual Daily Traffic, beginning 1977 dataset. The visualizations transform over 30 years, and over 230,000 rows of data into an accessible, consumable format for analysis and utilization. By selecting a municipality and route number, the average annual daily traffic for each segment along the route is displayed, showing average traffic volume changes over time.

New View Type – Data Lens

In 2015, New York State leveraged new functionality made available on the Open Data portal and introduced a new data view type called Data Lens. As with charts and graphs, data lens is another way to take existing data and present it in a visual way on Open NY. Data Lens moves beyond the display of static graphics, enabling interactive exploration and presentation of data in a new, visually engaging, and intuitive paradigm. It provides users the ability to search, view, and filter data on details of most interest to them. It synthesizes the data and displays analytical results through interactive visualizations (i.e., bar charts, maps, distribution graphs). The visualization below displays a view of the Data Lens page for Statewide 200kW or Less Residential/Non-Residential Solar Photovoltaic Incentive Program: Beginning 2000. Individual cards present the data geographically by installation location, temporally by installation date, and graphically by type of purchase and electric utility.

Open NY will continue to utilize technology to promote transparency, improve government performance, and enhance public engagement. It will also continue to deliver long-term growth and empower the public and government with data for the digital age. Open NY will continue to enhance user experience and interaction with the following:

• A new resources page, available through Open NY, including a library of formatting samples together with guidance materials to further standardization, interoperability, and utilization of open data within and outside of government.
• A new and engaging user interface for the data catalog;
• Advancing interactive visualizations and enhanced context to maximize understanding, analyses, and the utilization of data;
• Developing ecosystems and communities of practice with ongoing and continued expansion, and sharing publishable data;
• Expanding integration of Open NY in new applications and web portals; and
• Collaboration and innovation with the developer community.

Data is tightly woven into the fabric of tools and information we use to make decisions every day, as governments, businesses, and individuals. As Open NY continues to grow and evolve, we are committed to providing an exceptional customer experience and setting a new standard for data access and tools. We look forward to continuing our work with New York’s passionate data and technology community to achieve even greater transparency, access and innovation.

Transparency, Accessibility, Innovation, and Operational Excellence

Together these are the principles driving our performance