LLM Vulnerability Scanning with Garrick

This video provides a tutorial on using Garrick, a large language model vulnerability scanner, to test custom chat bots. Garrick employs probes (test cases) and detectors to identify vulnerabilities, drawing from a large, constantly updated probe library. The tutorial focuses on testing a simple chat application called Wy chat, which utilizes Llama 3 and the Grock API. It guides users through installing and configuring Garrick, emphasizing the “Generations” parameter for controlling request volume. The video also demonstrates how to identify API endpoints using browser developer tools, create JSON configuration files for REST API testing, and run tests with specific cases like malware generation.

The tutorial further explains how to analyze test results using hit logs and HTML reports. It highlights crucial security considerations, stressing the importance of proper authorization and avoiding testing systems without ownership, as Garrick can generate harmful content and create a high volume of requests. Overall, the video offers a practical guide to using Garrick for vulnerability scanning, particularly for custom chat bots, and underscores the need for responsible and ethical use of such tools.

Share this:

Like this:

Related