December 12, 2023 How Prompt-Based Development Revolutionizes Machine Learning Workflows Discover how prompt-based development can cut project timelines, reduce costs, and offer an agile approach to innovation. Georgi Naydenov Bobby Bahov In a previous blog post, we introduced a five-phase framework to plan out Artificial Intelligence (AI) and Machine Learning (ML) initiatives. We looked at an exemplary healthcare project with a potential problem of high readmission rates. We also discussed how to tailor AI to predict readmissions, offering a targeted solution that enhances patient care while effectively cutting costs and improving the patient’s well-being. Today, we delve into the traditional ML workflow and explore how prompt-based development accelerates the process by enabling parallel idea experimentation with minimal investment. The Traditional Machine Learning Workflow Initiating a traditional ML project begins with collecting data. In the healthcare scenario, this means collecting electronic medical records (EMR). This data then undergoes manual cleaning to address inconsistencies, from measurement outliers to data entry mistakes. Duplicated records are identified and rectified. Afterward, the data is labeled to create training and testing datasets. Following data engineering, we transition to algorithm and model selection, a choice influenced by both the business case and the type of available data. For instance, the decision between unsupervised and supervised learning, like opting for K-Means for cluster analysis or SVM for classification problems, plays a crucial role. The selected model may be trained on labeled datasets to effectively differentiate patients with high readmission risks from those with lower risks. Subsequently, data scientists evaluate the model’s accuracy, precision, and recall metrics to pinpoint high-risk patients. Once the model achieves optimal performance, it’s deployed to operational environments to forecast real-world hospital recurrence rates. The emphasis then shifts to collecting real-time patient insights. Incorporating these insights into the model’s dataset ensures its precision and relevancy by reflecting emerging health trends and new patient data. This comprehensive workflow forms the foundation of traditional ML processes. Traditional Machine Learning Workflow Challenges Challenges within the traditional ML workflow are not to be underestimated. Common iterative hurdles include: Data Scarcity — In most cases, model training requires a vast quantity of high-quality data. Its absence can cause compromises to the model’s performance and precision, affecting the reliability of the end product. Manual Annotation — An often-underestimated task is the manual labeling of data post-collection, where manual annotations are required to clarify the data. Model Retraining Costs — During the model training and evaluation phases, we might observe a deficiency in high-variance data, making it daunting to test our hypotheses. Leading to potential model retraining and increased costs in time and resources. These hurdles hinder the ability to validate assumptions and hypotheses, making such initiatives challenging without thorough planning. But what if we could develop the proof-of-concept (POC) in a fraction of the time? How Prompt-Based Development Aligns with Agility Prototyping ML models is notoriously difficult. Fortunately, the barrier to experimenting with multiple approaches is lowered by recent advancements in large language models (LLMs), coupled with the emerging field of prompt engineering. To draw a comparison, picture LLMs as a toolbox with tools for handling different activities and tasks. Prompt engineering is the manual detailing the tools you should use for a particular task. The more explicit your instructions (prompt), the more capable the toolbox (LLM) will be at solving your business needs. Leveraging the toolbox (LLMs) at our disposal in tandem with a manual (prompt engineering) brings us to a holistic approach to developing ML solutions called prompt-based development (PBD). How does this help us in practice? Prompt engineering introduces specialized techniques that allow a machine learning model to make accurate predictions with limited labeled data — zero-shot and few-shot prompting. Think back to the last time you were in a foreign country and came across a sign you couldn’t understand. Suddenly, you spot a familiar symbol that allows you to comprehend its message without knowing the language. Similarly, with zero-shot prompting, you prompt an LLM with no examples, which the model can still deduce, comprehend, and deliver based on the prompt and prior knowledge. Now, imagine you took language classes and learned common phrases. During your trip abroad, you came across similar sentences with minor changes, which you could still understand. That’s the equivalent of using a few-shot prompt for an LLM. With this method, you guide the model to solve a particular task, using some examples of similar tasks so you can improve its output. Prompt-Based Development Advantages In the realm of healthcare data curation, traditional methods involve laboriously identifying electronic medical records, removing duplicates, and annotating data — a process stretching over months. Enter prompt-based development (PBD), where the use of large language models (LLMs) and textual prompts streamlines and quickens this task significantly. This accelerated pace allows for early assessment of technical feasibility and prompt adjustments based on data quality. Contrast this with the rigidity of the traditional ML workflow, where changes in business requirements may necessitate retraining a model from scratch. By using PBD, we can instead change our prompting strategy and re-design the prompts used during development to accommodate changes to business requirements, enabling us to be more flexible and faster in our development approach. Furthermore, prompt-based development presents the opportunity for parallel hypothesis testing. We can simultaneously explore other assumptions besides validating our initial hypothesis on the correlation between readmission rates and discharge time. This may include other variables, such as treatment plan adjustments, comorbidities in patients, or the variability of seasonal fluctuations in their impact on hospital readmission rates. Once we have a firm grasp on the hypotheses, we can transition to deploying the model on a live environment in “shadow mode.” This guarantees the model’s operation without impacting real-world decisions, while its inferences are scrutinized and assessed against pre-defined expectations. The subsequent comparison of model metrics and outputs offers a pivotal moment. If the results are promising, we can consider revisiting the prototype model to develop it further into a more polished MVP and allow it to make real-world decisions. Alternatively, discrepancies in the model’s performance due to technical or ethical concerns present an opportunity to return to the drawing board and revisit our hypotheses. Ethical and Technical Considerations Although prompt-based development presents the opportunity for speedy delivery, shipping an ML model should be done responsibly. While it looks fantastic to build a model from scratch in a fraction of the time it usually takes, there are some crucial aspects to pay attention to, such as bias or unfairness. Ethically, it is crucial to reduce biases and stereotypes in the data to the utmost. For example, consider a data visualization that identifies lower-income neighborhoods as high risk for hospital readmission within 30 days post-discharge. Limiting ourselves only to that data source could reinforce harmful biases. Through the help of a subject matter expert, we can introduce new and relevant data sources and adjust our prompts to include further nuances such as access to primary care and the health check frequency, resulting in a better-informed and tailored approach to patient care. Furthermore, a subject matter expert is still required to assess the model’s performance before it can start making impactful decisions. This holds particular significance in healthcare applications and projects, especially in safeguarding a patient’s protected health information. However, the swift nature of prompt-based development shouldn’t compromise best practices. Rigorous testing, validation, and attention to technical details, such as code quality and library vulnerabilities, are indispensable to prevent incurring technical debt. Balancing speed with ethical and technical diligence is the key to unlocking the true potential of prompt-based development. Conclusion Prompt-based development is transforming machine learning initiatives, enabling us to hypothesize, experiment, and strategize with different approaches. It also significantly reduces project timelines, often at a fraction of the time and cost of the traditional approach. Each organization’s journey is unique, and the emergence of prompt-based development allows it to put its value propositions to the test and validate if it is one step closer to its strategic goal. Can your current or future ML projects benefit from quicker validation? If so, consider taking the PBD approach. Tags StrategyArtificial Intelligence Share Share on Facebook Share on LinkedIn Share on Twitter AI & Cyber Security Learn how AI augments human expertise to detect, prevent, and mitigate cyber threats effectively. Download Share Share on Facebook Share on LinkedIn Share on Twitter Sign up for our monthly newsletter. Sign up for our monthly newsletter.