Glossary of Key Terms

Artificial Intelligence (AI)
 A computer system that can simulate or perform human tasks.
Machine Learning (ML)
 A discipline within artificial intelligence (AI) dedicated to the development of algorithms that improve their performance on a human task without requiring explicit instructions.
ML Algorithm/Model
 A learned representation of the patterns inherent with the input data that can be used to generate predictions.
Pipeline
 A full sequence of steps to convert input data into output predictions. Typically involving loading, reformatting, and transforming data and predictions so that they can be integrating into real-time workflows.
Development Operations (DevOps)
 A philosophical framework combining best practices in information technology operations and software engineering to rapidly and robustly build and implement high-quality informatics solutions.
Machine Learning Operations (MLOps)
 A framework of technical best practices for deploying and maintaining machine learning applications efficiently and effectively.
Target/Ground-Truth
 The “gold-standard” definition to which ML pipeline predictions will be compared.
Discriminative Performance
 The degree to which predictions match the ground-truth labels, often measured by sensitivity, specificity, positive predictive value, and negative predictive value.
Implementation Efficacy
 The degree to which the final implementation of the ML pipeline satisfies the original need for which it was built.
Receiver Operating Characteristic (ROC Curve)
 A visualization of the trade-off between sensitivity and specificity across the full breadth of possible decision thresholds for a continuous output.
Class Imbalance
 The degree to which the proportion of class labels are skewed towards one label or the other.
Precision and Recall Curve
 A visualization of the trade-off between precision (positive predictive value) and recall (sensitivity) across the full breadth of possible decision thresholds for a continuous output.
F1 Score
 The harmonic mean of precision and recall. See more here.
Matthews Correlation Coefficient
 The Pearson correlation coefficient for two binary variables. See more here.
Cost-Sensitive Learning
 An learning approach where each classification error is assigned its own weight to fine-tune the model’s predilection towards certain error type1 2.
Equivocal Zone
 An interval of continuous output in which no binary class label is assigned.
Applicability Assessment
 The determination of whether a new input is similar enough to a model’s training data for a reliable prediction.
Demographic Parity
 A fairness criterion used to assess whether the outputs of a predictive model is independent of demographic groups (e.g. race, gender, or age).
Predictive Parity
 A fairness criterion used to assess whether the outputs of a model have equal positive and negative predictive values across demographic groups (e.g. race, gender, or age).
Equalized Odds
 A fairness criterion used to assess whether the outputs of a model have equal true positive rates (sensitivity) and false positive rates (specificity) across demographic groups (e.g. race, gender, or age).
Global Explainability
 The ability to estimate each feature’s impact on model outputs aggregated across an entire data set or feature space.
Local Explainability
 The ability to estimate each feature’s impact on model outputs for any given individual prediction.
Governance
 The processes by which organizational responsibilities and decisions are divided, evaluated, and executed.
Deployment
 Making a model or application accessible to other computers within a network.
Production Environment
 The software systems and infrastructure in which live applications are hosted and run for day-to-day operations.
Development Environment
 An isolated copy of the production environment where software changes can be tested without risk of impacting live operations.
Application-Programming Interface (API)
 A protocol or framework by which various software applications can communicate with each other and exchange data or predictions.
Human-in-the-Loop
 An implementation paradigm where model outputs are directed towards an expert user for incorporating into their decision-making before an action is taken.
Data Drift
 Divergence in input data away from the initial model training data set.
Concept Drift
 Divergence away from the training data in the target labels or context in which predictions are to be made.
Continuous Integration (CI)
 A DevOps principle in which changes to software are incorporate in small, manageable chunks continuously rather than large overhauls.
Continuous Deployment (CD)
 A DevOps principle in which updates to software a pushed to live environments without large periods of maintenance or down-time.

References

1.
Ling CX, Sheng VS. Cost-Sensitive Learning [Internet]. In: Sammut C, Webb GI, editors. Boston, MA: Springer US; 2011. p. 231–5.Available from: https://link.springer.com/10.1007/978-0-387-30164-8_181
2.
Mienye ID, Sun Y. Performance analysis of cost-sensitive learning methods with application to imbalanced medical data. Informatics in Medicine Unlocked [Internet] 2021;25:100690. Available from: https://linkinghub.elsevier.com/retrieve/pii/S235291482100174X