Recently, Elon Musk stated that licensing GPT-3 exclusively to Microsoft is a mistake.
This, of course, started gaining general media attention and both Microsoft and OpenAI are getting criticised for this decision, with the general mood being that exclusivity would harm AI's accessibility.
Broad access to artificial intelligence is of course a very important issue and one of the aims of the OpenAI consortium. However, being accessible is a very broad definition and ironically Microsoft's exclusivity could in many ways make GPT-3 more accessible than it would have been otherwise.
Counter-intuitive? Indeed it is. Let's explore some of the limits of GPT-3 and OpenAPI's accessibility:
GPT-3 was never going to be accessible to everyone:
OpenAPI publicly states that potential users are subject to a pre-approval process
GPT-3's training process is not fully reproducible:
Yes, you can see the code on github and the paper describes the data sources but anyone who works on data science knows very well that the data cleaning and optimisation process has a profound impact on model performance and that's not available publicly
GPT-3 deep learning training cost is beyond the means of most companies:
With a training cost alone estimated between $4M and $12M, it's obvious that it's priced well above what most companies can afford. Renting the API is going to be relatively affordable but there would be limits on control and customisation.
Finally, GPT-3 has extremely limited ability to explain its decision process, just like any other deep learning-based process:
Ironically, Elon himself warned about the dangers of unrestricted AI, but he seems blind about the issues of a black-box system.
This is not meant as criticism to the validity of OpenAPI's achievement, which is indeed technically impressive, but deep learning is clearly an increasingly capital-intensive road to AI, with serious limitations on transparency.
Truly accessible AI is still an unsolved problem that lacks a clear framework:
No commercial entity is likely to develop a truly transparent and easily reproducible process on its own for obvious reasons (among which the difficulty of keeping a profit margin on something easily copied), the development of accessible, transparent AI instead would require a broad, inter-disciplinary research work from academia to define a theoretical base and some advancements both in transparent machine learning and AI ethics.
While this article doesn't claim to have a solution, we can at least try to point at some very generic minimum requirements for a truly accessible artificial intelligence.
Truly accessible AI must be transparent in its decisions:
This is probably the most technically challenging point. The current state of the art forces a decision maker to choose between performance (especially on non-structured data) and transparency.
While there is on-going research on making deep learning decisions easier to explain, it still lacks the clarity of older classifiers such as decision trees.
The latter are much weaker in performance (and are nowadays used mostly as ensembles), especially on non-structured data, so further research is necessary in finding a more performing, transparent alternative.
Training set, data sources and any data treatment must be accessible:
Just like humans, machine learning algorithms are strongly influenced by the quality of the training set and this translates into inheriting both conscious and unconscious biases of whoever creates the training set. Xiaoice VS Zo is an excellent example of how training data influences AI behaviour.
To compensate for this issue, the entire data treatment process must be transparent to ensure that any potential bias and error in generating training variables can be detected and fixed by the broadest possible audience.
The algorithms must be trainable on consumer or pro-sumer-grade hardware:
Although this might seem a controversial point, if training an AI requires a multi-million dollar investment in hardware, then its use will always be restricted to a small circle of entities able to afford the cost, with everyone else forced to just rent it.
By limiting the hardware requirements to what can be obtained with an HEDT platform, this ensures that an effective AI could be trained and customised by smaller academic institutions and businesses, ensuring a fair competitive environment for all involved actors.
Ideally, as much of the process as possible should be done on a low-code environment:
While this is not strictly a requirement, the use of low-code principles ensures that a much broader audience could both audit and influence an AI training process.
Making the logical process flow accessible to non-coders could ensure that domain experts would be able to effectively have access to the AI development process instead of laying down a list of requirements and effectively delegating a programmer or data scientist into implementing it into code.
Microsoft is offering an alternative solution trough its machine teaching framework, however further simplification can be obtained by using existing software solutions.
Software like Knime or Alteryx can be used to create a visual logical flow and limit the code only to the necessary machine learning libraries instead of coding the entire ETL process.
All in all, Elon Musk might be right in being concerned about GPT-3 accessibility, however no matter how powerful GPT-3 is, it's not the accessible AI we're looking for.
No comments:
Post a Comment