)
报错ValueError: too many values to unpack (expected 2)复现RGCL模型遇到的问题该代码里面报错原因是collate_fn里的tokens是多层嵌套列表每个样本是分词后的 token 列表batch 打包后变成[[token1,token2...], [xxx]]但旧版 transformers3.1.0在is_pretokenizedTrue时内部解包逻辑不兼容或者数据里混入了非法空样本、维度错乱。解决方案原 collate_fndef collate_fn(data): u, i, r, tokens zip(*data) encoding bert_tokenizer(tokens, return_tensorspt, paddingTrue, truncationTrue, is_pretokenizedTrue) return torch.Tensor(u), torch.Tensor(i), torch.Tensor(r), \ encoding[input_ids], encoding[attention_mask]改成下面版本去掉is_pretokenizedTrue改用字符串拼接 / 直接传原文本彻底规避解包 bugdef collate_fn(data): u, i, r, token_lists zip(*data) # 把分词列表还原成字符串不用is_pretokenized text_list [ .join(toks) for toks in token_lists] encoding bert_tokenizer(text_list, return_tensorspt, paddingTrue, truncationTrue, max_lengthargs.review_max_length) return torch.Tensor(u), torch.Tensor(i), torch.Tensor(r), \ encoding[input_ids], encoding[attention_mask]infact,在修改代码之前我还多次尝试改变transformers的版本但是还是报一样的错误可能是没有改到正好可以用的那个版本。ps.之前跑自己的模型也遇到过版本问题修改transformers的版本就重新运行了完整报错(base) rootautodl-container-6d25459816-01b42b10:~/Work/ReviewGraph-main/BERT# python bert_whitening.py libgomp: Invalid value for environment variable OMP_NUM_THREADS 2026-07-06 00:23:43 - Load_Data - Start reading data to pandas. Clean string: 100%|███████████████████████████████████████████████████████████████████████| 963441/963441 [01:3600:00, 9971.91it/s] Delete unused words: 100%|██████████████████████████████████████████████████████████████| 963441/963441 [00:0800:00, 107299.97it/s] 2026-07-06 00:25:55 - Load_Data - Truncate review length to 56 words Delete unused words: 100%|██████████████████████████████████████████████████████████████| 963441/963441 [00:0300:00, 255554.45it/s] check data split: 770753it [00:33, 23069.12it/s] /root/Work/ReviewGraph-main/BERT/../load_data.py:190: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. train_data train_data.append([valid_data.loc[valid_drop_user_data_index], pre tokenize: 100%|███████████████████████████████████████████████████████████████████████| 973763/973763 [09:0400:00, 1789.32it/s] 0%| | 0/7608 [00:00?, ?it/s] Traceback (most recent call last): File bert_whitening.py, line 175, in module save_sentence_feat(args) File /root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py, line 27, in decorate_context return func(*args, **kwargs) File bert_whitening.py, line 149, in save_sentence_feat for u, i, r, input_ids, mask in tqdm(data_loader): File /root/miniconda3/lib/python3.8/site-packages/tqdm/std.py, line 1185, in __iter__ for obj in iterable: File /root/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py, line 530, in __next__ data self._next_data() File /root/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py, line 570, in _next_data data self._dataset_fetcher.fetch(index) # may raise StopIteration File /root/miniconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py, line 52, in fetch return self.collate_fn(data) File bert_whitening.py, line 100, in collate_fn encoding bert_tokenizer(tokens, return_tensorspt, paddingTrue, File /root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py, line 3021, in __call__ encodings self._call_one(texttext, text_pairtext_pair, **all_kwargs) File /root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py, line 3109, in _call_one return self.batch_encode_plus( File /root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py, line 3311, in batch_encode_plus return self._batch_encode_plus( File /root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils.py, line 886, in _batch_encode_plus ids, pair_ids ids_or_pair_ids ValueError: too many values to unpack (expected 2)
报错ValueError: too many values to unpack (expected 2)复现RGCL模型遇到的问题该代码里面报错原因是collate_fn里的tokens是多层嵌套列表每个样本是分词后的 token 列表batch 打包后变成[[token1,token2...], [xxx]]但旧版 transformers3.1.0在is_pretokenizedTrue时内部解包逻辑不兼容或者数据里混入了非法空样本、维度错乱。解决方案原 collate_fndef collate_fn(data): u, i, r, tokens zip(*data) encoding bert_tokenizer(tokens, return_tensorspt, paddingTrue, truncationTrue, is_pretokenizedTrue) return torch.Tensor(u), torch.Tensor(i), torch.Tensor(r), \ encoding[input_ids], encoding[attention_mask]改成下面版本去掉is_pretokenizedTrue改用字符串拼接 / 直接传原文本彻底规避解包 bugdef collate_fn(data): u, i, r, token_lists zip(*data) # 把分词列表还原成字符串不用is_pretokenized text_list [ .join(toks) for toks in token_lists] encoding bert_tokenizer(text_list, return_tensorspt, paddingTrue, truncationTrue, max_lengthargs.review_max_length) return torch.Tensor(u), torch.Tensor(i), torch.Tensor(r), \ encoding[input_ids], encoding[attention_mask]infact,在修改代码之前我还多次尝试改变transformers的版本但是还是报一样的错误可能是没有改到正好可以用的那个版本。ps.之前跑自己的模型也遇到过版本问题修改transformers的版本就重新运行了完整报错(base) rootautodl-container-6d25459816-01b42b10:~/Work/ReviewGraph-main/BERT# python bert_whitening.py libgomp: Invalid value for environment variable OMP_NUM_THREADS 2026-07-06 00:23:43 - Load_Data - Start reading data to pandas. Clean string: 100%|███████████████████████████████████████████████████████████████████████| 963441/963441 [01:3600:00, 9971.91it/s] Delete unused words: 100%|██████████████████████████████████████████████████████████████| 963441/963441 [00:0800:00, 107299.97it/s] 2026-07-06 00:25:55 - Load_Data - Truncate review length to 56 words Delete unused words: 100%|██████████████████████████████████████████████████████████████| 963441/963441 [00:0300:00, 255554.45it/s] check data split: 770753it [00:33, 23069.12it/s] /root/Work/ReviewGraph-main/BERT/../load_data.py:190: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. train_data train_data.append([valid_data.loc[valid_drop_user_data_index], pre tokenize: 100%|███████████████████████████████████████████████████████████████████████| 973763/973763 [09:0400:00, 1789.32it/s] 0%| | 0/7608 [00:00?, ?it/s] Traceback (most recent call last): File bert_whitening.py, line 175, in module save_sentence_feat(args) File /root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py, line 27, in decorate_context return func(*args, **kwargs) File bert_whitening.py, line 149, in save_sentence_feat for u, i, r, input_ids, mask in tqdm(data_loader): File /root/miniconda3/lib/python3.8/site-packages/tqdm/std.py, line 1185, in __iter__ for obj in iterable: File /root/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py, line 530, in __next__ data self._next_data() File /root/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py, line 570, in _next_data data self._dataset_fetcher.fetch(index) # may raise StopIteration File /root/miniconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py, line 52, in fetch return self.collate_fn(data) File bert_whitening.py, line 100, in collate_fn encoding bert_tokenizer(tokens, return_tensorspt, paddingTrue, File /root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py, line 3021, in __call__ encodings self._call_one(texttext, text_pairtext_pair, **all_kwargs) File /root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py, line 3109, in _call_one return self.batch_encode_plus( File /root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py, line 3311, in batch_encode_plus return self._batch_encode_plus( File /root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils.py, line 886, in _batch_encode_plus ids, pair_ids ids_or_pair_ids ValueError: too many values to unpack (expected 2)
 获取与实战)
双梁正问题 【 torch 实战】（Python代码实现）)
