Active learning (machine learning)

Active learning is a special case of machine learning in which a learning algorithm can interactively query a user (or some other information source) to label new data points with the desired outputs.[1][2][3] In statistics literature, it is sometimes also called optimal experimental design.[4] The information source is also called teacher or oracle.

This article is about a machine learning method. For active learning in the context of education, see active learning.
Part of a series on
Machine learning
and
data mining
Problems
Supervised learning
(classification • regression)
Clustering
Dimensionality reduction
Structured prediction
Anomaly detection
Artificial neural network
Reinforcement learning
Theory
Machine-learning venues
Glossary of artificial intelligence
Related articles

There are situations in which unlabeled data is abundant but manual labeling is expensive. In such a scenario, learning algorithms can actively query the user/teacher for labels. This type of iterative supervised learning is called active learning. Since the learner chooses the examples, the number of examples to learn a concept can often be much lower than the number required in normal supervised learning. With this approach, there is a risk that the algorithm is overwhelmed by uninformative examples. Recent developments are dedicated to multi-label active learning,[5] hybrid active learning[6] and active learning in a single-pass (on-line) context,[7] combining concepts from the field of machine learning (e.g. conflict and ignorance) with adaptive, incremental learning policies in the field of online machine learning.

Definitions

Let T be the total set of all data under consideration. For example, in a protein engineering problem, T would include all proteins that are known to have a certain interesting activity and all additional proteins that one might want to test for that activity.

During each iteration, i, T is broken up into three subsets

  1. : Data points where the label is known.
  2. : Data points where the label is unknown.
  3. : A subset of TU,i that is chosen to be labeled.

Most of the current research in active learning involves the best method to choose the data points for TC,i.

Scenarios
  • Membership Query Synthesis: This is where the learner generates its own instance from an underlying natural distribution. For example, if the dataset are pictures of humans and animals, the learner could send a clipped image of a leg to the teacher and query if this appendage belongs to an animal or human. This is particularly useful if the dataset is small.[8]
  • Pool-Based Sampling: In this scenario, instances are drawn from the entire data pool and assigned an informative score, a measurement of how well the learner “understands” the data. The system then selects the most informative instances and queries the teacher for the labels.
  • Stream-Based Selective Sampling: Here, each unlabeled data point is examined one at a time with the machine evaluating the informativeness of each item against its query parameters. The learner decides for itself whether to assign a label or query the teacher for each datapoint.

Query strategies

Algorithms for determining which data points should be labeled can be organized into a number of different categories, based upon their purpose:[1]

  • Balance exploration and exploitation: the choice of examples to label is seen as a dilemma between the exploration and the exploitation over the data space representation. This strategy manages this compromise by modelling the active learning problem as a contextual bandit problem. For example, Bouneffouf et al.[9] propose a sequential algorithm named Active Thompson Sampling (ATS), which, in each round, assigns a sampling distribution on the pool, samples one point from this distribution, and queries the oracle for this sample point label.
  • Expected model change: label those points that would most change the current model.
  • Expected error reduction: label those points that would most reduce the model's generalization error.
  • Exponentiated Gradient Exploration for Active Learning:[10] In this paper, the author proposes a sequential algorithm named exponentiated gradient (EG)-active that can improve any active learning algorithm by an optimal random exploration.
  • Uncertainty sampling: label those points for which the current model is least certain as to what the correct output should be.
  • Query by committee: a variety of models are trained on the current labeled data, and vote on the output for unlabeled data; label those points for which the "committee" disagrees the most
  • Querying from diverse subspaces or partitions:[11] When the underlying model is a forest of trees, the leaf nodes might represent (overlapping) partitions of the original feature space. This offers the possibility of selecting instances from non-overlapping or minimally overlapping partitions for labeling.
  • Variance reduction: label those points that would minimize output variance, which is one of the components of error.
  • Conformal Predictors: This method predicts that a new data point will have a label similar to old data points in some specified way and degree of the similarity within the old examples is used to estimate the confidence in the prediction.[12]
  • Mismatch-first farthest-traversal: The primary selection criteria is the prediction mismatch between the current model and nearest-neighbour prediction. It targets on wrongly predicted data points. The second selection criteria is the distance to previously selected data, the farthest first. It aims at optimizing the diversity of selected data.[13]

A wide variety of algorithms have been studied that fall into these categories.[1][4]

Minimum Marginal Hyperplane

Some active learning algorithms are built upon support-vector machines (SVMs) and exploit the structure of the SVM to determine which data points to label. Such methods usually calculate the margin, W, of each unlabeled datum in TU,i and treat W as an n-dimensional distance from that datum to the separating hyperplane.

Minimum Marginal Hyperplane methods assume that the data with the smallest W are those that the SVM is most uncertain about and therefore should be placed in TC,i to be labeled. Other similar methods, such as Maximum Marginal Hyperplane, choose data with the largest W. Tradeoff methods choose a mix of the smallest and largest Ws.

See also
  • List of datasets for machine learning research

Notes
  1. Settles, Burr (2010). "Active Learning Literature Survey" (PDF). Computer Sciences Technical Report 1648. University of Wisconsin–Madison. Retrieved 2014-11-18. Cite journal requires |journal= (help)
  2. Rubens, Neil; Elahi, Mehdi; Sugiyama, Masashi; Kaplan, Dain (2016). "Active Learning in Recommender Systems". In Ricci, Francesco; Rokach, Lior; Shapira, Bracha (eds.). Recommender Systems Handbook (2 ed.). Springer US. doi:10.1007/978-1-4899-7637-6. hdl:11311/1006123. ISBN 978-1-4899-7637-6. S2CID 11569603.
  3. Das, Shubhomoy; Wong, Weng-Keen; Dietterich, Thomas; Fern, Alan; Emmott, Andrew (2016). "Incorporating Expert Feedback into Active Anomaly Discovery". In Bonchi, Francesco; Domingo-Ferrer, Josep; Baeza-Yates, Ricardo; Zhou, Zhi-Hua; Wu, Xindong (eds.). IEEE 16th International Conference on Data Mining. IEEE. pp. 853–858. doi:10.1109/ICDM.2016.0102. ISBN 978-1-5090-5473-2. S2CID 15285595.
  4. Olsson, Fredrik (April 2009). "A literature survey of active machine learning in the context of natural language processing". SICS Technical Report T2009:06. Cite journal requires |journal= (help)
  5. Yang, Bishan; Sun, Jian-Tao; Wang, Tengjiao; Chen, Zheng (2009). "Effective multi-label active learning for text classification" (PDF). Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '09. p. 917. CiteSeerX 10.1.1.546.9358. doi:10.1145/1557019.1557119. ISBN 978-1-60558-495-9. S2CID 1979173.
  6. Lughofer, Edwin (February 2012). "Hybrid active learning for reducing the annotation effort of operators in classification systems". Pattern Recognition. 45 (2): 884–896. doi:10.1016/j.patcog.2011.08.009.
  7. Lughofer, Edwin (2012). "Single-pass active learning with conflict and ignorance". Evolving Systems. 3 (4): 251–271. doi:10.1007/s12530-012-9060-7. S2CID 43844282.
  8. Wang, Liantao; Hu, Xuelei; Yuan, Bo; Lu, Jianfeng (2015-01-05). "Active learning via query synthesis and nearest neighbour search" (PDF). Neurocomputing. 147: 426–434. doi:10.1016/j.neucom.2014.06.042.
  9. Bouneffouf, Djallel; Laroche, Romain; Urvoy, Tanguy; Féraud, Raphael; Allesiardo, Robin (2014). "Contextual Bandit for Active Learning: Active Thompson". In Loo, C. K.; Yap, K. S.; Wong, K. W.; Teoh, A.; Huang, K. (eds.). Neural Information Processing (PDF). Lecture Notes in Computer Science. 8834. pp. 405–412. doi:10.1007/978-3-319-12637-1_51. ISBN 978-3-319-12636-4. HAL Id: hal-01069802.
  10. Bouneffouf, Djallel (8 January 2016). "Exponentiated Gradient Exploration for Active Learning". Computers. 5 (1): 1. arXiv:1408.2196. doi:10.3390/computers5010001. S2CID 14313852.
  11. "shubhomoydas/ad_examples". GitHub. Retrieved 2018-12-04.
  12. Makili, Lázaro Emílio; Sánchez, Jesús A. Vega; Dormido-Canto, Sebastián (2012-10-01). "Active Learning Using Conformal Predictors: Application to Image Classification". Fusion Science and Technology. 62 (2): 347–355. doi:10.13182/FST12-A14626. ISSN 1536-1055. S2CID 115384000.
  13. Zhao, Shuyang; Heittola, Toni; Virtanen, Tuomas (2020). "Active learning for sound event detection". arXiv:2002.05033 [eess.AS].

Other references
  • Active Learning Tutorial, S. Dasgupta and J. Langford.
  • ALMA (Software) Open source tool for implementing online active learning
  • modAL (Software) Spen source Modular Active Learning toolkit
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.