feat: add amd support

[ParthJadhav]

No description provided.

[Yobb17]

https://www.remotasks.com/internal/login/facebook/callback?code=AQD1EJXqfiRkFIUj53lIxSjOEsFGQ8WnzPzFML-uQsoMfnUsmKMZ3vpvm-N2DcCcNVJsfTgjtqYVvb5rMr8.

[AnttiRae]

Hey! I'm not sure if I did this correctly but here's my output from running ./run.sh --model 7b --with-rocm. It seems that something went wrong with docker not detecting my GPU. Let me know if there's something more specific I should test.
I'm running Fedora 38 with AMD RX 7900 XTX as the GPU.

Edit: I tried with Windows wsl2 (ubuntu) as well and got the same error.

[cotsuka]

Finally found time to test this PR. Looks like I'm running into DNS resolution issues hitting several of the repos. I ran the same command as @AnttiRae above. I'll attempt to retest later to see if that clears up.

[cweiske]

Works here.

Speed depends on the GPU; here my GPU is slower than the CPU.
CPU: AMD Ryzen 7 7700, 16 cores with 64GiB RAM
GPU: AMD Radeon RX 7600, 8GB

Sample request timings with 70b:

llama-gpt-api-rocm-ggml-1  | llama_print_timings:       total time = 225636.50 ms
llama-gpt-api-1            | llama_print_timings:       total time = 160374.53 ms