Skip to content

Update README.md #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 29, 2024
Merged

Update README.md #32

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 13 additions & 63 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,77 +32,27 @@ hackingBuddyGPT is described in [Getting pwn'd by AI: Penetration Testing with L
## Existing Agents/Usecases

We strive to make our code-base as accessible as possible to allow for easy experimentation.
Our experiments are structured into `use-cases`, e.g., privilege escalation attacks. A researcher
wanting to create a new experiment would just create a new use-case that mostly consists
of the control loop and corresponding prompt templates. We provide multiple helper and base
classes, so that a new experiment can be implemented in a few dozens lines of code as
connecting to the LLM, logging, etc. is taken care of by our framework. For further information (esp. if you want to contribute use-cases), please take a look at [docs/use_case.md](docs/use_case.md).

Our experiments are structured into `use-cases`, e.g., privilege escalation attacks, allowing Ethical Hackers to quickly write new use-cases (agens).

Our initial forays were focused upon evaluating the efficiency of LLMs for [linux
privilege escalation attacks](https://arxiv.org/abs/2310.11409) and we are currently breaching out into evaluation
the use of LLMs for web penetration-testing and web api testing.

### Privilege Escalation Attacks

How are we doing this? The initial tool `wintermute` targets linux priv-esc attacks. It uses SSH to connect to a (presumably) vulnerable virtual machine and then asks OpenAI GPT to suggest linux commands that could be used for finding security vulnerabilities or privilege escalation. The provided command is then executed within the virtual machine, the output fed back to the LLM and, finally, a new command is requested from it..

#### Current features (wintermute):

- connects over SSH (linux targets) or SMB/PSExec (windows targets)
- supports OpenAI REST-API compatible models (gpt-3.5-turbo, gpt4, gpt-3.5-turbo-16k, etc.)
- supports locally running LLMs
- beautiful console output
- logs run data through sqlite either into a file or in-memory
- automatic root detection
- can limit rounds (how often the LLM will be asked for a new command)

#### Example run

This is a simple example run of `wintermute.py` using GPT-4 against a vulnerable VM. More example runs can be seen in [our collection of historic runs](docs/old_runs/old_runs.md).

![Example wintermute run](docs/example_run_gpt4.png)

Some things to note:

- initially the current configuration is output. Yay, so many colors!
- "Got command from LLM" shows the generated command while the panel afterwards has the given command as title and the command's output as content.
- the table contains all executed commands. ThinkTime denotes the time that was needed to generate the command (Tokens show the token count for the prompt and its response). StateUpdTime shows the time that was needed to generate a new state (the next column also gives the token count)
- "What does the LLM know about the system?" gives an LLM generated list of system facts. To generate it, it is given the latest executed command (and it's output) as well as the current list of system facts. This is the operation which time/token usage is shown in the overview table as StateUpdTime/StateUpdTokens. As the state update takes forever, this is disabled by default and has to be enabled through a command line switch.
- Then the next round starts. The next given command (`sudo tar`) will lead to a pwn'd system BTW.

#### Academic Publications on Priv-Esc Attacks

Preliminary results for the linux privilege escalation use-case can be found in [Evaluating LLMs for Privilege-Escalation Scenarios](https://arxiv.org/abs/2310.11409):

~~~ bibtex
@misc{happe2024llms,
title={LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks},
author={Andreas Happe and Aaron Kaplan and Jürgen Cito},
year={2024},
eprint={2310.11409},
archivePrefix={arXiv},
primaryClass={cs.CR}
}
~~~

This work is partially based upon our empiric research into [how hackers work](https://arxiv.org/abs/2308.07057):

~~~ bibtex
@inproceedings{Happe_2023, series={ESEC/FSE ’23},
title={Understanding Hackers’ Work: An Empirical Study of Offensive Security Practitioners},
url={http://dx.doi.org/10.1145/3611643.3613900},
DOI={10.1145/3611643.3613900},
booktitle={Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
publisher={ACM},
author={Happe, Andreas and Cito, Jürgen},
year={2023},
month=nov, collection={ESEC/FSE ’23}
}
~~~
| Name | Description | Screenshot |
| -- | -- | -- |
| minimal | A minimal 50 LoC Linux Priv-Esc example | |
| [linux-privesc](docs/linux_privesc.md) | Given a SSH-connection for a low-privilege user, task the LLM to become the root user. This would be a typical Linux privilege escalation attack. We published two academic papers about this: [paper #1](https://arxiv.org/abs/2308.00121) and [paper #2](https://arxiv.org/abs/2310.11409) | ![Example wintermute run](docs/example_run_gpt4.png) |
| web-pentest | Directly hack a webpage | |
| web-api-pentest | An Web-API focues usecase | |

## Build your own Agent/Usecase

A researcher
wanting to create a new experiment would just create a new use-case that mostly consists
of the control loop and corresponding prompt templates. We provide multiple helper and base
classes, so that a new experiment can be implemented in a few dozens lines of code as
connecting to the LLM, logging, etc. is taken care of by our framework. For further information (esp. if you want to contribute use-cases), please take a look at [docs/use_case.md](docs/use_case.md).

The following would create a new (minimal) linux privilege-escalation agent. Through using our infrastructure, this already uses configurable LLM-connections (e.g., for testing OpenAI or locally run LLMs), logs trace data to a local sqlite database for each run, implements a round limit (after which the agent will stop if root has not been achieved until then) and is able to connect to a linux target over SSH for fully-autonomous command execution (as well as password guessing).

~~~ python
Expand Down