Home Data Engineering Data Education Explaining Exploratory Data Analysis

Explaining Exploratory Data Analysis

Audio version of the article

Discovered in the 1970s by American mathematician John Tukey, exploratory data analysis (EDA) is a method of analysing and investigating the data sets to summarise their main characteristics. Scientists often use data visualisation methods to discover patterns, spot anomalies, check assumptions or test a hypothesis through summary statistics and graphical representations.

EDA goes beyond the formal modelling or hypothesis to give maximum insight into the data set and its structure, and in identifying influential variables. It can also help in selecting the most suitable data analysis technique for a given project. Specific knowledge, such as the creation of a ranked list of relevant factors to be used as guidelines, can also be obtained using EDA.

Types of EDA

The EDA types of techniques are either graphical or quantitative (non-graphical). While the graphical methods involve summarising the data in a diagrammatic or visual way, the quantitative method, on the other hand, involves the calculation of summary statistics. These two types of methods are further divided into univariate and multivariate methods.

Univariate methods consider one variable (data column) at a time, while multivariate methods consider two or more variables at a time to explore relationships. Thus, there are four types of EDA in all — univariate graphical, multivariate graphical, univariate non-graphical, and multivariate non-graphical. The graphical methods provide more subjective analysis, and quantitative methods are more objective.

  • Univariate non-graphical: This is the simplest form of data analysis among the four options. In this type of analysis, the data that is being analysed consists of just a single variable. The main purpose of this analysis is to describe the data and to find patterns.
  • Univariate graphical: Unlike the non-graphical method, the graphical method provides the full picture of the data. The three main methods of analysis under this type are histogram, stem and leaf plot, and box plots. The histogram represents the total count of cases for a range of values. Along with the data values, the stem and leaf plot shows the shape of the distribution. The box plots graphically depict a summary of minimum, first quartile median, third quartile, and maximum.
  • Multivariate non-graphical: The multivariate non-graphical type of EDA generally depicts the relationship between multiple variables of data through cross-tabulation or statistics.
  • Multivariate graphical: This type of EDA displays the relationship between two or more set of data. A bar chart, where each group represents a level of one of the variables and each bar within the group represents levels of other variables.

EDA Tools

Python and R language are the two most commonly used data science tools to create an EDA.

Python: EDA can be done using python for identifying the missing value in a data set. Other functions that can be performed are — the description of data, handling outliers, getting insights through the plots. Due to its high-level, built-in data structure, and dynamic typing and binding make it an attractive tool for EDA. Analyzing a dataset is a hectic task that takes a lot of time. Python provides certain open-source modules that can automate the whole process of EDA and help in saving time.

R: The R language is used widely by data scientists and statisticians for developing statistical observations and data analysis. R is an open-source programming language which provides a free software environment for statistical computing and graphics that is supported by the R Foundation for Statistical Computing.

Wrapping Up

Apart from the functions described above, EDA can also:

  • Perform k-means clustering. It is an unsupervised learning algorithm where the data points are assigned to clusters, also known as k-groups. K-means clustering is commonly used in market segmentation, image compression, and pattern recognition.
  • EDA can be used in predictive models such as linear regression, where it is used to predict outcomes.
  • It is also used in univariate, bivariate, and multivariate visualization for summary statistics, establishing relationships between each variable, and for understanding how different fields in the data interact with each other.

This article has been published from a wire agency feed without modifications to the text. Only the headline has been changed.

Source link

- Advertisment -

Most Popular

Understanding Nearest Centroids linear classification in Machine Learning

Nearest Centroids is a linear classification machine learning algorithm. It involves predicting a class label for new examples based on which class-based centroid the example...

Development careers that use Python

Over the past few decades, several programming languages ​​have been invented, and many have gained space in the development market. JavaScript, C ++, Swift,...

Data Science Strategies for post Trump Era

Data can be a source of comfort and confidence as the pandemic rages on and the Trump Administration comes to a close. Now that the...

Upcoming opportunities in Crypto

A record-breaking year for DeFi, but can it break away from the negative perceptions around yield farming and pump-and-dump schemes? While 2020 will go down...

Introductory Guide on XCFramework and Swift Package

In WWDC 2019, Apple announced a brand new feature for Xcode 11; the capability to create a new kind of binary frameworks with a special format...

Understanding Self Service Data Management

https://dts.podtrac.com/redirect.mp3/www.dataengineeringpodcast.com/podlove/file/704/s/webplayer/c/episode/Episode-159-Isima.mp3 Summary The core mission of data engineers is to provide the business with a way to ask and answer questions of their data. This often...
- Advertisment -