Generative AI: Supporting AI Literacy, Research, and Publishing

Generative AI tools used in research can be:

Useful for:

• Defining well-known terms and topics
• Starting research, for things where initial accuracy isn’t critical (you just need some keywords and ideas), that you know well, or that are well-described (though  be careful because of “hallucinations” - basically all LLMs lie) - FACT CHECK!!!
• Identifying top scholars in a field or subject area, especially if you’re looking for diverse scholars
• Brainstorming topic ideas, keywords, subject headings or possible databases to search 
• Narrowing down your research question. The more specific you are with your prompts, the better
• Summarizing and simplifying dense text (though its accuracy and utility may vary)
• Translating common languages
• Generating audiovisuals and code
• Classifying or analyzing large datasets or organizing info

Not recommended for:

• Use if prohibited by your instructor, PI or faculty mentor, the journals you or your collaborators plan to submit to, or other stakeholders
• Current (or future) events or things that change quickly unless it’s connected to Internet (e.g. ChatGPT is only trained on data up to Sept. 2021 but Bing Chat and Bard will give you search links)
• Topics that you don’t know well or where accuracy is critical without verifying from multiple sources (because of “hallucinations” - basically all LLMs lie) - FACT CHECK!!!
• Topics that are ill-defined, obscure, non-mainstream, non-Western, or personal - basically things that don’t have a lot of information online about them or that  AI can’t “know” (How to Get an AI to Lie To You)
• Sensitive or potentially harmful or offensive topics
• Anything private or confidential
• Generating whole papers
• Generating citations and bibliographies (unless it's connected to a source for literature like Scite.ai)

In general, remember that these tools are  often a black box and likely biased.  Large language models (LLMs) also are known to “hallucinate” (e.g. lie and make things up); they are designed to give you the statistically most  likely next words or output - prioritizing fluency over accuracy (GPT 4 is better but not perfect). They also don’t know anything about the real world. Remember, these are “stochastic parrots” (Bender et. al), not all-knowing magic beings or anything like a human intelligence no matter how fluent they are!

Not all AI research tools are generative AI tools like ChatGPT. Many of the tools below may be using some combination of machine learning, deep learning, natural language processing and more to find and visualize the research literature. Note: Many of these tools are experimental and are in beta (or alpha) so consider them pilots, not the last word for comprehensive searching. All of these are freely available or have a free tier. For more information on some of these see this presentation.

Visual Search and Visualization Tools

Other AI-powered discovery and visualization tools can be:

Useful for:

• Finding relevant papers you might not ordinarily have found by means other than keyword matching, either by searching similar or seeding relevant papers. NOTE: systems that  work from seed papers are often better than those that ask you topics you’re interested in or work like recommender systems (e.g. Amazon “you might also like”), though none of them are perfect
• Visualizing networks of citations and researchers

Not recommended for:

• Comprehensive searches or those requiring exact replication. Uses these as a supplement to library databases and Google Scholar 
• Humanities and non-empirical social sciences may be much less well represented depending on corpus of text the search is based on and results vary by field and subfield
• Interdisciplinary research (except for Inciteful), especially for recommender-based systems