Abstract
ChatGPT Search was launched on October 31 by OpenAI as a new AI-powered search engine. Among its features, it stands out for its ability to retrieve information from various online sources, including scholarly databases, which potentially allows the use of this tool for academic tasks, both quantitative and qualitative.
To test its features, five academic tasks are designed: two quantitative (collecting hit count estimates from Google Search and scraping bibliometric indicators from ResearchGate); two qualitative tasks (performing a narrative synthesis of an academic topic and generating a brief academic author profile), and a mixed task (identifying, collecting and describing a list of publications from Google Scholar Profiles).
The results show the inability of ChatGPT Search to conduct quantitative tasks correctly, fabricating the results (hallucination). Qualitative tasks are performed with better results; however, errors are detected, which prevent recommending the tool without manual analysis and refinement. Finally, the ability to generate links to scientific publications can open up competition among academic sites to be mentioned in the ChatGPT Search responses, giving rise to Academic Generative Engine Optimization (A-GEO).
Keywords
Generative artificial intelligence; AI; Link analysis; Web search; Search engines; Narrative synthesis; Information retrieval; ChatGPT Search; Scholarly tasks; Academic tasks; Quantitative vs. Qualitative; Academic Generative Engine Optimization (A-GEO).
Introduction
On July 25, 2024, the company OpenAI launched the prototype of a new AI web search engine called SearchGPT, accessible to 10,000 users for testing.
This prototype combined language models from the GPT-4 family with real-time web access to organize and interpret information, rather than simply providing a flat list of links. This approach enabled users to receive answers to their queries quickly, in a contextualized format, and based on relevant sources, as reported in the official documentation.
While the term «SearchGPT» was used during the prototype phase, the integrat-ed feature is finally known as «ChatGPT Search», officially launched on October 31, 2024, and initially available for ChatGPT Plus and ChatGPT Team users
The search engine is operational for the GPT-4oand GPT-4o mini language families, both online (via chatgpt.com) and on desktop (Windowsand macOS) and Apps (iOS, Android). Additionally, ChatGPT Search can be installed as the default search engine in the browser through the plugin available in the Chrome Web Store.
The launch of ChatGPT Search represents a clear strategic move to position itself in the new AI web search engine market, a strategy that Meta is also following in order not only to reduce its dependence on traditional web search engines that are transitioning to the AI market, such as Google (with Gemini) or Bing (Copilot), but also to compete directly with other native AI search engines, such as Perplexity.
The responses generated by ChatGPT Search include inline web citations through a “Sources” button, which, when clicked, opens a side menu that con-tains the references used in generating the response, along with a link to the original content.
Additionally, ChatGPT Search provides the kind of output characteristic of generative AI, that is, narrative syntheses in response to prompts. Furthermore, this response process is conducted in the absence of advertising.
The tool also incorporates follow-up questions, non-textual answers (i.e., imag-es and videos), as well as real-time information from the Web, provided by its own web crawler, (OAI Searchbot), used to discover and surface websites in search results. Additionally, ChatGPT Search leverages third-party search providers and their partners’ content to provide comprehensive information to the final users.
(…)
Download and useful links
How to cite
Font-Julián, C. I., Orduña-Malea, E., & Codina, L. (2024). ChatGPT Search as a tool for scholarly tasks: evolution or devolution?. Infonomy, 2(5). https://doi.org/10.3145/infonomy.24.059
References
Aggarwal, Pranjal; Murahari, Vishvak; Rajpurohit, Tanmay; Kalyan, Ashwin; Narasimhan, Karthik; Deshpande, Ameet (2024). Geo: Generative engine optimization. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 5-16). New York (USA): ACM. https://arxiv.org/pdf/2311.09735
Bains, Callum (2024). The chatbot optimisation game: Can we trust AI web searches? The Guardian, 3 November. https://www.theguardian.com/technology/2024/nov/03/the-chatbot-optimisation-game-can-we-trust-ai-web-searches
Lopezosa, Carlos; Goyanes, Manuel; Codina, Lluís (2024) Acelerando la investigación cualitativa con inteligencia artificial: una guía práctica para el diseño, desarrollo y ejecución de investigación con entrevistas. Barcelona: Universitat de Barcelona. http://hdl.handle.net/2445/211948
Lopezosa, Carlos; Rovira, Cristòfol; Codina, Lluís (2024). La IA generativa y su aplicación al SEO, SEM, analítica web y diseño UX. Barcelona: Universitat Pompeu Fabra, 92 pp. https://doi.org/10.31009/cr.2024.11
Orduña-Malea, Enrique; Alonso-Arroyo, Adolfo (2017). Cybermetric techniques to evaluate organizations using web-based data. Oxford: Chandos Publishing.
https://doi.org/10.1016/C2016-0-00399-6
Thelwall, Mike (2010). Webometrics: Emergent or doomed? Information Research: An International Electronic Journal, v. 15, n. 4. https://informationr.net/ir/15-4/colis713.html
Thelwall, Mike; Stuart, David (2006). Web crawling ethics revisited: Cost, privacy, and denial of service. Journal of the American Society for Information Science and Technology, v. 57, n. 13, pp. 1771–1779. https://doi.org/10.1002/asi.20388
Urbano, Cristóbal (2024). Los chatbots de IA generativa como fuentes de tráfico web: aproximación desde la analítica web de un portal de revistas. Anuario ThinkEPI, v.18. https://doi.org/10.3145/thinkepi.2024.e18a31
Wan, Alexander; Wallace, Eric; Klein, Dan (2024). What evidence do language models find convincing? arXiv preprint. https://arxiv.org/pdf/2402.11782
To know more
- Other IA entries for this site (mainly in Spanish, but some in English)