在Grace Hopper上安装

1. 安装 TensorRT-LLM（已在 Ubuntu 22.04 上测试）。

   pip3 install torch==2.5.1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
   
   sudo apt-get -y install libopenmpi-dev && pip3 install tensorrt_llm
   

   如果使用PyTorch NGC Container镜像，则不需要执行安装支持CUDA的PyTorch包的先决步骤。

2. 通过在Python中运行以下内容来检查安装是否正常（已在Python 3.10上测试）：

   from tensorrt_llm import LLM, SamplingParams
   
   prompts = [
       "Hello, my name is",
       "The president of the United States is",
       "The capital of France is",
       "The future of AI is",
   ]
   sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
   
   llm = LLM(model="TinyLlama/TinyLlama-1.1B-Chat-v1.0")
   
   outputs = llm.generate(prompts, sampling_params)
   
   # Print the outputs.
   for output in outputs:
       prompt = output.prompt
       generated_text = output.outputs[0].text
       print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")