Hugging Face's logo
Maurice Weber's picture
56

Maurice Weber

mauriceweber
ilyayudkovich's profile picture Nosdivad's profile picture thomwolf's profile picture
·

AI & ML interests

None yet

Recent Activity

New activity about 13 hours ago
togethercomputer/RedPajama-Data-V2
authored a paper 1 day ago

Organizations

Together's profile picture Together AI Ablations's profile picture

mauriceweber's activity

New activity in togethercomputer/RedPajama-Data-V2 about 13 hours ago

Add paper citation

1
#30 opened 1 day ago by davanstrien
New activity in togethercomputer/RedPajama-Data-V2 3 months ago

RPV2 ccnet preprocessing

1
#29 opened 3 months ago by bpwl0121
New activity in togethercomputer/RedPajama-Data-1T-Sample 4 months ago

sample split details

3
#4 opened about 1 year ago by sujantkumarkv
New activity in togethercomputer/RedPajama-Data-V2 5 months ago

How can I download the sample-10B fastestly?

1
#28 opened 5 months ago by zgxiao
New activity in togethercomputer/RedPajama-Data-1T 7 months ago

defunct book subset

4
#28 opened about 1 year ago by polinaeterna
New activity in togethercomputer/RedPajama-Data-V2 7 months ago

How much disk space would the whole HF dataset take?

1
#27 opened 7 months ago by protossw512
New activity in togethercomputer/RedPajama-Data-V2 10 months ago

rpv2-subsamples

1
#26 opened 11 months ago by mauriceweber

The doc_id in duplicates is should contain?

3
#24 opened 11 months ago by newbietuan
New activity in togethercomputer/RedPajama-Data-V2 11 months ago

Deduplication steps

23
#15 opened about 1 year ago by ilyayudkovich

Here's a download script parallelized using Spark

1
#22 opened 11 months ago by srowen

what is the meaning of snapshots in redpajama-data-v2?

2
#21 opened 11 months ago by choidonghun

How to join documents and quality signals when downloading directly

3
#19 opened 12 months ago by tgshdyfuhuf
New activity in togethercomputer/RedPajama-Data-V2 12 months ago

Missing duplicates parquet files

5
#18 opened 12 months ago by bebensee

Script to download all files of 1B sample data locally

2
#13 opened about 1 year ago by ivanzhouyq

What is the total size, of the entirety of this dataset in TB?

1
#10 opened about 1 year ago by Bayaz

What's the concept on partitions

2
#5 opened about 1 year ago by SwatCat

quality_signals, minhash and duplicates missing

2
#3 opened about 1 year ago by sheshanshag

Request to add retries into RedPajama-Data-V2.py script

1
#16 opened about 1 year ago by yura38

How to obtain duplicates from minhash?

1
#8 opened about 1 year ago by cq
New activity in togethercomputer/RedPajama-Data-V2 about 1 year ago

Obtaining Filtered Samples

4
#12 opened about 1 year ago by ssingh22