The Essential Role of Promptbooks in Microsoft Copilot for Security

In this blog, we will explore the concept of promptbooks, their role in the promptbook library, and why these tools are essential for using Microsoft Copilot for Security (CfS) effectively. We will dive into how promptbooks can streamline and enhance your security operations, enabling more efficient and consistent results. From understanding how CfS works, to the benefits and practical applications of promptbooks, and the specifics of Microsoft’s prebuilt promptbooks, we will cover everything you need to know to leverage these tools effectively in your security workflows.

What Are Promptbooks?

If you are reading this blog, then you already have some familiarity with promptbooks, so I won’t bore you with another definition. Instead, I’ll provide a link to the official definition from Microsoft for reference.

Based on my experience using the new capabilities provided by CfS in limited laboratory settings, I can confidently say that without promptbooks, CfS would be impossible to use effectively. If you are new to using large language models (LLMs) or generative AI engines or are more of an early adopter, using something like ChatGPT-4 versus CfS is like night and day. Do not expect to prompt CfS with long, complex prompts with vague objectives. Do not expect it to write perfect emails or reports; rather, expect it to get you to all your various data faster and with more context and correlation.

How Copilot for Security Works

When a prompt is given to CfS, it first decides which plugin (preinstalled or custom) to use to get the best response. If you already know which plugin you want to use—in other words, what data you want to access—it is best to include that information in your prompt. This helps CfS quickly identify the appropriate plugin, saving time in the decision-making process. Sometimes, CfS may choose the wrong plugin, or a different one based on your past sessions. Thus, it's best practice to create effective prompts that can be saved and reused within promptbooks to ensure consistent responses, a topic we will explore further when we discuss the necessity of promptbooks. It is also critical to preinstall any custom plugins you may want to use before developing your promptbooks.

CfS decides which plugin to use by evaluating the plugin’s definition file that is read into the LLM. When a prompt is given, it matches the prompt's language with the best match found in the plugin definitions. If those plugin definitions are not well-developed or the language/context of the prompt is unclear, the engine may use the wrong plugin. After the plugin is chosen, CfS will then begin to process the data to formulate a response. Simply put, CfS chooses the plugin, processes the data it finds, and then generates a response. In my experience, response quality can vary even when using identical prompts, often because CfS may choose the wrong plugin for a particular session, causing significant variation in the responses. Additionally, there are quality issues related to formatting. For example, when requesting a summary report on a particular topic, certain unidentified factors can cause the report's format to be inferior to previous experiences. It feels like a slot machine sometimes, you never know what you will get.

What are Promptbooks good for?

Promptbooks are useful for several reasons. They allow for sequential execution, where prompts are executed in a specific order, with each prompt building on the response from the previous one. They automate repetitive and time-consuming tasks, enabling security professionals to focus on more critical activities. Additionally, users can create custom promptbooks tailored to their unique security needs, ensuring the tool adapts to various scenarios.

What is the Promptbook Library?

The Promptbook Library is a repository within Microsoft Copilot for Security that houses both prebuilt and custom promptbooks. It serves as a central hub for managing, deploying, sharing and collaborating around promptbooks. Beyond the built-in promptbook library, there are also many searchable examples hosted on sites like GitHub that you should familiarize yourself with to get the most out of promptbooks. For example, customers can access the official GitHub repository for CfS at aka.ms/CopilotForSecurityGithub, where they will find some example promptbooks. Additionally, other repositories, such as Rod Trent’s repository, are worth exploring and can be found here.

Microsoft prebuilt promptbooks

Presently there are five Microsoft specific or prebuilt promptbooks available in CfS, four of which are documented, starting here explaining the Microsoft Sentinel incident investigation promptbook. Quickly, the five prebuilt promptbooks are as follows:

1. Microsoft 365 Defender incident investigation: Enter a Defender incident ID for an incident report with related alerts, reputation scores, users and devices (7 prompts).

2. Microsoft Sentinel incident investigation: Enter a Sentinel incident ID for an incident report with related alerts, reputation scores, users and devices (7 prompts).

3. Suspicious script analysis: Enter a script snippet for analysis around intent, intelligence, threat actors and impacts (6 prompts).

4. Threat actor profile: Enter any threat actor name and receive a profile of their known tools and tactics along with protection suggestions (5 prompts).

5. Vulnerability impact assessment: Enter any CVEID then receive vulnerability intelligence and mitigation strategies (4 prompts).

These five promptbooks serve as a good introduction to proper promptbook development and provide value, particularly in summarizing common questions around vulnerabilities, threat actors, and incident analysis using Microsoft Sentinel and Defender, if you are using those toolsets. If you are not using those tools, it is likely that your large security vendor has already published a custom plugin or will soon. Keep in mind that not all plugins are equal. Depending on the investment the vendor put into the plugin, it may turn out to be less effective than you initially envisioned, so manage your expectations when testing a custom plugin. If your favorite security vendor hasn’t published one, ask them about it. Additionally, remember that each prompt incurs some cost, so using this tool efficiently, effectively, and consistently is crucial to making it more useful than costly. With a pay-as-you-go model, costs can quickly escalate if CfS is used for activities that might be better served by a basic web search, a free LLM, or a different Microsoft Copilot product such as Outlook for emails or Word for enhanced reporting capabilities.

Why Are Promptbooks Necessary?

Simply put, and as I touched on earlier, CfS can be temperamental for lack of a better descriptor. It isn’t as forgiving as OpenAI's ChatGPT or Microsoft Copilot for Bing and Edge. CfS wasn’t designed or built to be an all-encompassing LLM; it is a narrowly trained LLM that excels in cybersecurity topics. It can interpret data from your vast array of sources, both cloud-based and on-premises, to correlate information for faster and more efficient decision-making. This contrasts with manually performing tasks across multiple platforms, systems, and access points.

Promptbooks enhance efficiency and automation by streamlining operations and ensuring consistent security procedures, thereby reducing human error. They enable comprehensive and systematic analysis of security incidents, allowing for rapid threat identification and mitigation. Customizable and scalable, promptbooks can be tailored to address specific security challenges and adapt to evolving needs. By integrating seamlessly with Microsoft security tools like Sentinel and Defender Suite, promptbooks provide a cohesive and unified approach to cybersecurity, centralizing operations within the Microsoft ecosystem and beyond. However, you’ll need to determine if the benefits outweighs the costs.

I know it is tacky to quote yourself, but as I started this blog, I will conclude it: “I can confidently say that without promptbooks, CfS would be impossible to use effectively.”