Skip to content

RuntimeError: CUDA error: #21

@Usaywook

Description

@Usaywook

When I tried to implement this code with the below command, I got this error.

  • command
python Evaluation.py ./config/demo.py --model-name demo
  • error
gpu number:1
model loaded from ./models/demo.pt
Successfully Loaded model: ./models/demo.pt
Finished Initialization in 15.365s!!!
  0%|                                                                                                                                                                                                                                                            | 0/1234 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "Evaluation.py", line 77, in <module>
    out = model(data)
  File "/home/usaywook/anaconda3/envs/mmTrans/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/usaywook/Samsung_T5/tmp/mmTransformer/lib/models/mmTransformer.py", line 150, in forward
    social_mask, lane_enc, lane_mask)
  File "/home/usaywook/anaconda3/envs/mmTrans/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/usaywook/Samsung_T5/tmp/mmTransformer/lib/models/TF_version/stacked_transformer.py", line 128, in forward
    lane_mem = self.lane_enc(self.lane_emb(lane_enc), lane_mask) # (batch size, max_lane_num, 128)
  File "/home/usaywook/anaconda3/envs/mmTrans/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/usaywook/Samsung_T5/tmp/mmTransformer/lib/models/TF_utils.py", line 49, in forward
    x = layer(x, x_mask)
  File "/home/usaywook/anaconda3/envs/mmTrans/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/usaywook/Samsung_T5/tmp/mmTransformer/lib/models/TF_utils.py", line 69, in forward
    x = self.sublayer[0](x, lambda x: self.self_attn(x, x, x, mask))
  File "/home/usaywook/anaconda3/envs/mmTrans/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/usaywook/Samsung_T5/tmp/mmTransformer/lib/models/TF_utils.py", line 208, in forward
    return x + self.dropout(sublayer(self.norm(x)))
  File "/media/usaywook/Samsung_T5/tmp/mmTransformer/lib/models/TF_utils.py", line 69, in <lambda>
    x = self.sublayer[0](x, lambda x: self.self_attn(x, x, x, mask))
  File "/home/usaywook/anaconda3/envs/mmTrans/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/usaywook/Samsung_T5/tmp/mmTransformer/lib/models/TF_utils.py", line 170, in forward
    query, key, value, mask=mask, dropout=self.dropout)
  File "/media/usaywook/Samsung_T5/tmp/mmTransformer/lib/models/TF_utils.py", line 227, in attention
    scores = torch.matmul(query, key.transpose(-2, -1)) / math.sqrt(d_k)
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)`

If I had removed .cuda in line 61 and line 75 from this code, I could resolve the error.
However, I cannot use the GPU to implement this code.

Moreover, in this repository, I cannot find the loss function to consider multimodal trajectories. Could you share the code for the loss function used in the original paper?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions