XPK Start: Thu Apr 16 23:26:59 UTC 2026 PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. 2026-04-16 23:27:23.160459: E external/local_xla/xla/stream_executor/cuda/cuda_platform.cc:51] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: UNKNOWN ERROR (303) I0416 23:27:23.688106 136687315519296 max_utils.py:273] Attempting to initialize the jax distributed system... I0416 23:27:32.727760 136687315519296 distributed.py:149] Starting JAX distributed service on [::]:8482 I0416 23:27:32.730163 136687315519296 distributed.py:172] Connecting to JAX distributed service on mt-16-zarr3-false-sv-qxzdd-slice-job-0-0.mt-16-zarr3-false-sv-qxzdd:8482 I0416 23:27:34.141583 136687315519296 max_utils.py:284] Jax distributed system initialized! Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/deps/src/maxtext/trainers/pre_train/train.py", line 852, in <module> app.run(main) File "/usr/local/lib/python3.12/site-packages/absl/app.py", line 367, in run _run_main(main, args) File "/usr/local/lib/python3.12/site-packages/absl/app.py", line 312, in _run_main sys.exit(main(argv)) ^^^^^^^^^^ File "/deps/src/maxtext/trainers/pre_train/train.py", line 844, in main config, recorder, diagnostic_config = initialize(argv) ^^^^^^^^^^^^^^^^ File "/deps/src/maxtext/trainers/pre_train/train.py", line 773, in initialize config = pyconfig.initialize(argv) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/deps/src/maxtext/configs/pyconfig.py", line 294, in initialize pydantic_config = initialize_pydantic(argv, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/deps/src/maxtext/configs/pyconfig.py", line 426, in initialize_pydantic pydantic_config = types.MaxTextConfig(**pydantic_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/pydantic/main.py", line 263, in __init__ validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pydantic_core._pydantic_core.ValidationError: 1 validation error for MaxTextConfig profiler Input should be '', 'xplane' or 'nsys' [type=enum, input_value=None, input_type=NoneType] For further information visit https://errors.pydantic.dev/2.13/v/enum XPK End: Thu Apr 16 23:27:39 UTC 2026 EXIT_CODE=1