Data science allows us to extract insights and knowledge from data, driving decision-making and innovation in business, health care, entertainment, and beyond. The field has roots in math, statistics, and computer science, but it only began to emerge as its own distinct field in the early 2000s with the proliferation of digital data and advances in computing power and technology. It gained significant momentum and recognition around the mid to late 2000s with the rise of big data and the need for sophisticated techniques to analyze and derive insights from large and complex datasets. Its evolution since then has been rapid, and it is quickly becoming a cornerstone of many industries and domains.
Data, however, is not new! Humans have been collecting data and generating datasets from the beginning of time. This started in the Stone Age when people carved shapes and pictures, called petroglyphs, on rock. Petroglyphs provide insights into how animals looked and how people carried out their daily lives, offering valuable historical “data.” Ancient Egyptians invented papyrus—an early form of paper—to journal their data. Papyrus made it easier to store data in bulk, such as listing inventories, recording financial transactions, and preserving stories for future generations.
The word data is the plural of the Latin word datum, which translates as “something that is given or used.” It is often used to mean a single piece of information or a single point of reference in a dataset.
When you hear the word “data,” you may think of numbers. While numbers are a primary form of data, there are many other forms of data all around us. Anything that we can analyze to compile information—high-level insights—is considered data.
Quantitative Data: Deals with quantities, amounts, and numeric measurements. It is highly structured and naturally suited for mathematical calculations and statistical analysis. Quantitative variables are divided into two main types:
Continuous Data: Can take any value within a range, including infinite fractions and decimals. Continuous data must be measured rather than counted (for example, there is an infinite number of possible values between 0 and 0.1, such as 0.05, 0.001, or 0.0001).
Discrete Data: Takes numerical values that follow specific intervals, making it directly countable. Examples include shoe sizes (e.g., 5, 5.5, 6, 6.5, 7), where 5.5 is valid, but 5.25 is not.
Categorical Data: Represented in various forms such as words, symbols, or code numbers. The possible values a categorical variable can take are called its levels. Unlike numeric data, categorical values do not indicate a physically measurable quantity. Instead, values are chosen from a finite set of classes:
Nominal Data: Unordered categorical variables where the set of possible values carries no natural order. Examples include state of residence (AL, AK, WY), binary indicators (Yes/No), or experimental groups (treatment/control).
Ordinal Data: Categorical variables where the levels possess a natural, logical ordering, even if the exact mathematical distance between values cannot be measured. Examples include letter grades (A, B, C, D, F), education levels (high school diploma, some college, bachelor’s degree), or satisfaction ratings (“Not recommended at all” to “Highly recommended”).
Qualitative Data: Consists of non-numerical data that describe subjective attributes, experiences, or characteristics. Unlike categorical data which is cleanly structured into predefined categories, qualitative data often deals with unquantifiable, complex details. Examples include raw descriptions, field observations, personal interviews, open-ended survey responses, or photos.
A dataset is a collection of observations or data entities organized for analysis and interpretation. Many datasets can be represented as a table where each row indicates a unique data entity and each column defines the structure of those entities.
The dataset in Table 4.1.1 has six entities (also referred to as items, entries, or instances), distinguished by semester. Each entity is defined by a combination of four attributes or characteristics (also known as features or variables)—Semester, Instructor, Class Size, and Rating.
Although the actual values of the attributes vary across entities, all entities have values for the same four attributes, making this a structured dataset. Structured datasets are organized in a tabular format with clearly defined fields and relationships.
By contrast, an unstructured dataset lacks a predefined or organized data model. Unstructured data lacks a fixed schema and is often in the form of text, images, videos, audio recordings, or other content where information does not fit neatly into rows and columns.
Unstructured data is extremely common—some estimates suggest that most real-world data is unstructured. Examples include product reviews, social media posts, public images, and short online videos. In practice, these unstructured datasets are often processed into structured formats so data scientists can analyze them effectively, as discussed in Section 4.4.
