But this software dominance is also slowly being challenged. OpenAI developed Triton, an open-source software solution that it claims is more efficient than CUDA. Triton can only be used on Nvidia’s GPUs as of now.97 Meta developed PyTorch and then spun off the project as an open-source initiative housed under the Linux Foundation (still financially supported by Meta), and its new version performs relatively well on Nvidia’s A100.98 The benefit of PyTorch is that it can be used across a range of hardware, but on the flip side, it is not optimized for any particular chip

Code for processing data samples can get messy and hard to maintain; we ideally want our dataset code to be decoupled from our model training code for better readability and modularity.

Tensors on the CPU and NumPy arrays can share their underlying memory locations, and changing one will change the other.

For example, if you are using TensorFlow, you might save your model as a .h5 file using the Keras API. If you are using PyTorch, you might save your model as a .pt file using the torch.save() function. By saving your model as a file, you can easily load it into a deployment environment (such as FastAPI) and use it to make predictions on new images

def compare_state_dict(dict1, dict2): # compare keys for key in dict1: if key not in dict2: return False for key in dict2: if key not in dict1: return False for (k,v) in dict1.items(): if not torch.all(torch.isclose(v, dict2[k])) return False return True

PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) { m.def("forward", &lltm_forward, "LLTM forward"); m.def("backward", &lltm_backward, "LLTM backward"); }

可能在返回Tensor底层数据中使用了新的内存

需要先使用 t2 的 stride (1, 4) 转换到 t2 的结构，再基于 t2 的结构使用 stride (1,) 转换为形状为 (12,)的 b 。但这不是view工作的方式，view 仅在底层数组上使用指定的形状进行变形，即使 view 不报错，它返回的数据是

Tensor底层一维数组元素的存储顺序与Tensor按行优先一维展开的元素顺序是否一致。

torch.view等方法操作需要连续的Tensor

peterjc123 / packages / pytorch

Each tensor has a .grad_fn attribute that references a Function that has created the Tensor (except for Tensors created by the user - their grad_fn is None).

A replacement for numpy to use the power of GPUs