Abstract
In this work we consider the problem of data classification in post-classical settings where the number of training examples consists of mere few data points. We explore the phenomenon and reveal key relationships between dimensionality of AI model’s feature space, non-degeneracy of data distributions, and the model’s generalisation capabilities. The main thrust of our present analysis is on the influence of nonlinear feature transformations mapping original data into higher- and possibly infinite-dimensional spaces on the resulting model’s generalisation capabilities. Subject to appropriate assumptions, we establish new relationships between properties of nonlinear feature transformation maps and the probabilities to learn successfully from few presentations.
I.Y. Tyukin—The work was supported by the UKRI Turing AI Fellowship EP/V025295/2 and the UKRI Trustworthy Autonomous Systems in Verifiability node EP/V026801/2.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bartlett, P.L., Long, P.M., Lugosi, G., Tsigler, A.: Benign overfitting in linear regression. Proc. National Acad. Sci. 117(48), 30063–30070 (2020)
Gorban, A.N., Grechuk, B., Mirkes, E.M., Stasenko, S.V., Tyukin, I.Y.: High-dimensional separability for one-and few-shot learning. Entropy 23(8), 1090 (2021)
Gorban, A.N., Tyukin, I.Y., Prokhorov, D.V., Sofeikov, K.I.: Approximation with random bases: pro et contra. Inf. Sci. 364–365, 129–145 (2016)
Kainen, P.C., Kůrková, V.: Quasiorthogonal dimension. In: Kosheleva, O., Shary, S.P., Xiang, G., Zapatrin, R. (eds.) Beyond Traditional Probabilistic Data Processing Techniques: Interval, Fuzzy etc. Methods and Their Applications. SCI, vol. 835, pp. 615–629. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-31041-7_35
Kainen, P.C., Kurkova, V.: Quasiorthogonal dimension of Euclidian spaces. Appl. Math. Lett. 6(3), 7–10 (1993)
Ledoux, M.: The concentration of measure phenomenon. Am. Math. Soc. Number 89 (2001)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, pp. 4077–4087 (2017)
Tyukin, I.Y., Gorban, A.N., Alkhudaydi, M.H., Zhou, Q.: Demystification of few-shot and one-shot learning. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2021)
Tyukin, I.Y., Gorban, A.N., Grechuk, B., Green, S.: Kernel stochastic separation theorems and separability characterizations of kernel classifiers. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2019)
Tyukin, I.Y., Gorban, A.N., Green, S., Prokhorov, D.: Fast construction of correcting ensembles for legacy artificial intelligence systems: algorithms and a case study. Inf. Sci. 485, 230–247 (2019)
Tyukin, I.Y., Gorban, A.N., McEwan, A.A., Meshkinfamfard, S., Tang, L.: Blessing of dimensionality at the edge and geometry of few-shot learning. Inf. Sci. 564, 124–143 (2021)
Tyukin, I.Y., Gorban, A.N., Sofeykov, K.I., Romanenko, I.: Knowledge transfer between artificial intelligence systems. Front. Neurorobotics 12, 49 (2018)
Vapnik, V.N.: An overview of statistical learning theory. IEEE Trans. Neural Netw. 10(5), 988–999 (1999)
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, pp. 3630–3638 (2016)
Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016)
Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64(3), 107–115 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tyukin, I.Y., Sutton, O., Gorban, A.N. (2023). Learning from Few Examples with Nonlinear Feature Maps. In: Arai, K. (eds) Intelligent Computing. SAI 2023. Lecture Notes in Networks and Systems, vol 711. Springer, Cham. https://doi.org/10.1007/978-3-031-37717-4_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-37717-4_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-37716-7
Online ISBN: 978-3-031-37717-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)