Fall 2026, I'll be offering yet another all-new PhD Seminar in AI for Business and Society!

QST IS911: LLM Agents in the Wild: Dissecting, Steering, and Incepting New Business and Social Science Research
Nearly four years after LLMs entered mainstream use, they have evolved from simple chatbots into tool-using agents deployed in the wild, interacting with consumers, organizations, and other agents. At the same time, recent CS research continues to uncover distinctive peculiarities, limitations, and failure modes of LLMs and agentic systems. This seminar begins by covering cutting-edge CS research to understand these systems in depth, including their theory, algorithms, and empirical behavior. We then turn to emerging research in business, economics, and the social sciences that describes, documents, and theorizes the frictions and opportunities arising from real-world deployment—a deep dive into LLMs in the wild. The goal is to equip students with the technical and conceptual foundations needed to identify new thesis-worthy research streams in business and the social sciences. We will cover three modules: mechanistic interpretability, which dissects the inner workings of LLMs and explores how they may be steered and controlled; peculiarities of frontier foundational models, which examine emerging capabilities and failure modes; and agent-to-agent interaction, where agents may be humans, LLMs, or both. The seminar will also have a special section on the role of AI in science and cover how to use state-of-the-art AI tools for research and personal productivity.
 
Previous iterations of this seminar have consistently identified and explored topics well ahead of their mainstream adoption by the public and their emergence within the mainstream business and social science research communities, earning a reputation for staying ahead of the curve: Interpretable Machine Learning and Bias in ML (2017), Generative AI in Business (2019), Neural Language Models and the Economics of AI (2020), Generative AI & Causal Inference with Text (2023), Agentic LLMs & Multi-Agent Systems For Business Research (2025). Collectively, these seminars have contributed to the publication or on-going working papers of over 35 journal articles in top-tier business journals, such as PNAS, Science Advances, Management Science, JMR, ISR, MISQ, SMJ, Marketing Science, and alongside numerous contributions to leading conferences like CHI, AAAI, ACL, ICML, EMNLP, Neurips, and KDD.
​

---

All Courses Created From Scratch and Taught (Except for 1)

My MSBA students conducted mock trials based on ongoing cases such as NYT vs. OpenAI and Musk vs. OpenAI. I was amazed by their ability to present compelling arguments, execute realistic trial procedures, leverage precedent-backed reasoning, and forecast how judicial decisions might establish new precedents. Extremely proud of their work—better TV than Boston Legal or Suits!

2023 Generative AI Doctoral Seminar

2025 Data Ethics: Responsible AI :)

2023 Responsible AI for Business Analytics Class :)

2019 Mining Unstructured Data: Deep Learning Based Text Mining (MSBA & MBA)

80%~90% percent of current exponential data growth is attributed to unstructured data such as text, pictures, and videos. Increasingly, the data has become more like crude oil that has to be refined and structured to extract value for business insights. Managers and data scientists now must understand and be able to refine unstructured data for competitive advantage. In this hands-on introductory class, you will learn the basic concepts and techniques of data mining unstructured data in addition to when and how they can be applied to improve many aspects of business and consumers’ welfare. In particular, we focus on text mining in this class, as text data in business is still one of the largest and the most important source of insight. Throughout the course, we will use Python, a powerful language and the main tool used by deep learning data scientists. For deep learning, we use Tensorflow + Keras deep learning framework, which is widely used for simplicity and great community support. You will learn a variety of text mining algorithms to automatically extract concepts such as Latent Dirichlet Allocation, Word2Vec to learn word semantics through neural networks, as well as deep learning approaches to usual tasks such as sentiment analysis and classification tasks. We end the course with increasingly important topic of interpretable machine learning and explainable artificial intelligence concerned with making sense out of machine learning predictions for use in business and society.
This course taught in 2019 obtained 5 out of 5 - Below is the evaluation.

Your browser does not support viewing this document. Click here to download the document.

2017 Interpretable ML & Bias in ML (PhD)

This is a seminar class intended to familiarize doctoral students on research approach that follows the Economining principle where it applies machine learning techniques such as deep learning, natural language processing, and computer vision along with rich economic theories to analyze interesting questions in business research. The first half of the class will be dedicated to reading and discussing recent advances in machine learning including topics in Deep Learning and Deep Reinforcement Learning. The second half of the class will be dedicated to reading and discussing business research papers that apply these methods to solve real-world business problems. This year, we focus on TAKE ML for Business: Transparent, Accountable, Kind, and Efficient ML for Business. Curated papers in CS/Stat on investigating bias in algorithms, fairness in algorithms, improving interpretability of algorithms will be covered and discussed. 
Keywords: Bias in Algorithm, Interpretable ML for Business, Deep Learning for Business Research. ​

2016 Unstructured Data & Big Data: Acquisition to Analysis (PhD)

This course has two parts. The first part will familiarize PhD students with cutting-edge tools and resources available to them for empirical research. This part of the course is primarily concerned with acquiring, cleaning, storing, and managing data for empirical business research, and it will provide hands-on experience using a variety of modern computing tools. Topics such as scraping data, using APIs, cluster computing, contacting companies, and cleaning data with regex will be covered among other things. The second part will start with a brief overview of machine learning concepts, techniques, text mining, and end with introduction to deep learning. At the end of the course, students will know what tools are most appropriate for different data analysis jobs at hand. And more importantly, the student should have obtained or started the process of obtaining unique and interesting datasets worthy of top journal research. We will use Python throughout the course.
Keywords: Data Collection, Dealing with Companies & NDA, Basic Text Mining, Starting Empirical Business Research.

2015-2021: Data Mining (MBA) 45851​

Interest in big data analytics has skyrocketed recently. The recent explosion in large-scale high-resolution data enables managers to ask and answer questions regarding businesses and consumers at a whole new level. Managers are faced with data about businesses and consumers that are growing faster than they can be utilized. Data mining enables business to extract useful consumer behavior and preferences from seemingly tremendous and unorganized data, which then can be utilized for data-driven decision-making and competitive advantage. Applications can be found in e-commerce, sales, marketing, finance, operations, etc. In this hands-on introductory class, you will learn the basic concepts and techniques of data mining (decision tree, naive bayes classifier, SVM, etc.) and introductory text mining in addition to when and how they can be applied to improve many aspects of business and consumers' welfare. We will discuss the marvelous power of data mining concepts and tools applied to data via troves of current real-world examples. Throughout the course, we will use a cutting-edge statistical tool in data mining and business analytics.

2015-2017 Data Mining and Business Analytics (Undergraduates)  - Created and Handed-off

Created and handed off. I now teach a version of this at the MBA level (DATA MINING (MBA) 45851​) only. 

2014-2015 Tech Camp  - Big Data, Right Tools: Computational Resources for Empirical Research (PHD)

This course eventually evolved into Unstructured Data & Big Data: Acquisition to Analysis above. ​I created and taught this course in my doctoral program (UPenn) for other incoming PhDs. I realized that doctoral students needed more programming and technical help to jumpstart empirical research. So I developed and proposed this course at the meeting of department heads. It was approved and I taught the course for 2 years. The “Tech Camp” is still active.

2019 Generative Models (PHD)

This is a seminar class intended to familiarize doctoral students on cutting-edge empirical techniques in Machine Learning to enable asking interesting questions in business research. We focus on generative models this year.
Keywords: Generative model

2020 Neural Language Models & Economics of AI  (PHD)​

This seminar covers state-of-the-art neural NLP models such as transformer, GPT families, etc. Additionally, selected Econ of AI topics are covered.

2022- current ​QST IS 811 RESPONSIBLE AI FOR BUSINESS ANALYTICS (MBAS, PEMBAS)

Wednesday 6:30-9:15, March 16-May 4 (except April 20), HAR 412.
AI and data are powerful tools that require thoughtful and responsible applications. The stakes are high as algorithms can scale to influence millions and even billions of people within a moment of launch and morph businesses and society with the impact that might last generations. Our mission is to understand and anticipate potential pitfalls and misuse of personal data and AI to guard against harm. We will also discuss existing solutions and the lack of solutions. The course is designed to be accessible to everyone and targeted to students whose businesses will use some form of data and algorithms. 1.5 CR

2022 - current QST BA 840 Data Ethics (MSBA)

This class examines ethical issues of data, data science, and algorithms. We consider unintended consequences and transparency of algorithms, phenomena such as mass personalization and experimentation, and examine competing ideas about privacy and the sometimes blurry line between the private and the public spheres in the digital age. The course is intended to place analytics in a social context and equip students to anticipate and understand the ethical tradeoffs they will be making in the process of doing analytical work.

2023 QST BA 820 Unstructured and Unsupervised Machine LEarning

​80%~90% percent of current exponential data growth is attributed to unstructured data such as text, images, audio, and video. Increasingly, the data has become more like crude oil that has to be refined and structured to extract value for business insights and strategies. Managers need to understand the opportunities and challenges associated with unstructured data for competitive advantage. In this class, students will learn what businesses can do with unstructured data through variety of case-based examples based on research and industry applications from Marketing, Information Systems, Finance, Strategy, and Social Impact perspectives. This is a course dedicated to understanding the potentials of unstructured data in different settings curated based on techniques involved. We first examine how to recognize patterns hidden in data by utilizing unsupervised learning for exploration and visualization. Then we examine how to structure unstructured data with a specific focus on text data.

2023 Generative AI and Causal Inference with Text (PHD)

2025 Agentic LLMs & Multi-Agent Systems For Business Research (PHD)

​We will explore the latest advances and empirical methods in LLMs, Agentic LLMs, Multi-Agentic Systems (MAS), and other cutting-edge research, including their application in fields such as business, economics, sociology, and psychology. The seminar emphasizes the practical use and investigation of these technologies within both business and societal contexts. Our focus is on the responsible application of these systems, the development of innovative algorithms to optimize business processes and inform managerial strategies, and the exploration of solutions to the unintended consequences and unforeseen frictions that arise in real-world settings.
 
The primary goal of this course is to inspire scholars to explore and pursue novel, high-impact research in business and computational social sciences. Key topics include agentic systems and creativity, emerging challenges in business and social science research using LLMs, opportunities and obstacles associated with Agentic Systems and MAS, and their integration into computational social research. As part of the course, students will be required to propose a new research idea relevant to their field of study, drawing inspiration from the seminar's content. The goal should not be to just deliver final project but to publish the idea in a top journal eventually. Additionally, the goal is not merely to inspire a single new paper but to catalyze new streams of interdisciplinary research (for your future tenure packet).

2026 LLM Agents in the Wild: Dissecting, Steering, and Incepting New Business and Social Science Research (PHD)

​Nearly four years after LLMs entered mainstream use, they have evolved from simple chatbots into tool-using agents deployed in the wild, interacting with consumers, organizations, and other agents. At the same time, recent CS research continues to uncover distinctive peculiarities, limitations, and failure modes of LLMs and agentic systems. This seminar begins by covering cutting-edge CS research to understand these systems in depth, including their theory, algorithms, and empirical behavior. We then turn to emerging research in business, economics, and the social sciences that describes, documents, and theorizes the frictions and opportunities arising from real-world deployment—a deep dive into LLMs in the wild. The goal is to equip students with the technical and conceptual foundations needed to identify new thesis-worthy research streams in business and the social sciences. We will cover three modules: mechanistic interpretability, which dissects the inner workings of LLMs and explores how they may be steered and controlled; peculiarities of frontier foundational models, which examine emerging capabilities and failure modes; and agent-to-agent interaction, where agents may be humans, LLMs, or both. The seminar will also have a special section on the role of AI in science and cover how to use state-of-the-art AI tools for research and personal productivity.
 
Previous iterations of this seminar have consistently identified and explored topics well ahead of their mainstream adoption by the public and their emergence within the mainstream business and social science research communities, earning a reputation for staying ahead of the curve: Interpretable Machine Learning and Bias in ML (2017), Generative AI in Business (2019), Neural Language Models and the Economics of AI (2020), Generative AI & Causal Inference with Text (2023), Agentic LLMs & Multi-Agent Systems For Business Research (2025). Collectively, these seminars have contributed to the publication or on-going working papers of over 35 journal articles in top-tier business journals, such as PNAS, Science Advances, Management Science, JMR, ISR, MISQ, SMJ, Marketing Science, and alongside numerous contributions to leading conferences like CHI, AAAI, ACL, ICML, EMNLP, Neurips, and KDD.