{"answer": "Less well-off", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:0", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:0", "question": "According to the report, how do 5% of the Latinos see economic upward mobility for their children?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Latinos interviewed by cellphone", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1", "question": "According to the report, which one is greater in population in the survey? Foreign born Latinos, or the Latinos interviewed by cellphone?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Some college or more", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:2", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:2", "question": "From this report, which subgroup among Hispanics has gained most confidence from 2008 to 2015?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}, {"region": {"page": 22}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}, {"region": {"page": 23}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:3", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:3", "question": "Among all 12 references in this report, how many are from its own research center?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:4", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:4", "question": "Among all charts in this report, how many compare between the general public with the Latinos/Hispanics? A few small charts sharing the same chart title is considered one chart.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Poor Financial Condition", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:5", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:5", "question": "According to this report, among the Hispanics shaped by four current personal financial situation, which situation involves the highest percentage that expects their future financial situation to get a lot worse?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['White', '10%']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:PH_2016.06.08_Economy-Final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:6", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:6", "question": "According to this report, from 2014 to 2015, one group has the most significant drop of percentage of households claiming their income was falling behind cost of living.\nWhich group is this, and what is the percentage droped?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "18.29%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:Independents-Report.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:Independents-Report.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:7", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:7", "question": "What's the percentage of people who are democrats and voted in the last election compared to the entire population in 2018?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:8", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:8", "question": "What's the percentage of people who are democrats and voted in the last election compared to the entire population in 2024?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:9", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:9", "question": "From this report, among Clinton, G.W.Bush, and Obama, whose support rate by the democrats increased by 5% in his last year of presidency compared to first year?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "no leans who are male", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:Independents-Report.pdf"}, {"region": {"page": 22}, "source_id": "mmlongbench:Independents-Report.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:10", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:10", "question": "According to this report, which group is greater in population in 2018? Republican who are hispanic, or no leans who are male?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:Independents-Report.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:Independents-Report.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:11", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:11", "question": "According to this report, how many charts provide no opinions only from the \"no lean\" group?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:Independents-Report.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:Independents-Report.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:Independents-Report.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:Independents-Report.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:Independents-Report.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:Independents-Report.pdf"}, {"region": {"page": 22}, "source_id": "mmlongbench:Independents-Report.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:12", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:12", "question": "How many charts and tables in this report are sourced from Annual totals of Pew Research Center survey data?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Berlin School of Experimental Psychology", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:0e94b4197b10096b1f4c699701570fbf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:13", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:13", "question": "Where was Gestalt psychology concieved?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "21%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:0e94b4197b10096b1f4c699701570fbf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:14", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:14", "question": "According to the chart on page 14 how much time was spent with family and friends in 2010?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0-375 miles", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:0e94b4197b10096b1f4c699701570fbf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:15", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:15", "question": "What range does red color represents in approximate distance from the Mississippi River in the chart that tracks the West NIle Virus in Europe?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Elements of objects tend to be perceptually grouped together if they form a pattern that is regular, simple, and orderly.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:0e94b4197b10096b1f4c699701570fbf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:16", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:16", "question": "How does this document define the law of good gestalt?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Eating out", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:0e94b4197b10096b1f4c699701570fbf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:17", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:17", "question": "Which category has the most increase from 2005 to 2010 for time spent on weedends?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Europe", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:0e94b4197b10096b1f4c699701570fbf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:18", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:18", "question": "Which continent has the most number of registered participant for advanced science course in CTBTO?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Circle', 'Rectangle']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:0e94b4197b10096b1f4c699701570fbf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:19", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:19", "question": "Which shapes does the slide use to illustrate the closure aspect of Gestalt Principle of Visual Perception?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "The centres of Indian Space Programme", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:20", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:20", "question": "What does the map in the report shows?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2015-2016", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:21", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:21", "question": "What year is the report for?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Agricultural Drough', 'Forest Fire', 'Landslides and Earthquakes monitoring', 'Groundwater Prospects Mapping', 'Inventory', 'Monitoring of Glacial Lakes/Water Bodies', 'Sericulture Development', 'Satellite Aided Search and Rescue']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:22", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:22", "question": "What are the uses of Indian remote sensing satellite constellation?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "National Atmospheric Research Laboratory", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:23", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:23", "question": "What is NARL stand for in this document?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Space Science & Planetary Exploration Satellites.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:24", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:24", "question": "Of the four missions of Indian Space Program's space mission from 2012 to 2017, which mission includes the launch of least number of satellites?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "19", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:25", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:25", "question": "How many organisations does this report introduce in detail (at least one paragraph) ?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "10", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:fdac8d1e9ef56519371df7e6532df27d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:26", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:26", "question": "How many exterior photos of organisations does this report provide?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Basic flat & layered maps", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:52b3137455e7ca4df65021a200aef724.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:27", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:27", "question": "What is the title of module 1?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "https://w.wiki/6gb", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 47}, "source_id": "mmlongbench:52b3137455e7ca4df65021a200aef724.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:28", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:28", "question": "What is the web address in the orange box in page 47 that gives the Wiki Query Service code?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "purple", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 81}, "source_id": "mmlongbench:52b3137455e7ca4df65021a200aef724.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:30", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:30", "question": "What is the color of the zone Mali in the map used to demonstrate GeoShapes using SPARQL and OSM?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "WikidataMapMakingWorkshop.ipynb", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 101}, "source_id": "mmlongbench:52b3137455e7ca4df65021a200aef724.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:31", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:31", "question": "Which example notebook does module 3 use to show how to create an interactive map?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 24}, "source_id": "mmlongbench:52b3137455e7ca4df65021a200aef724.pdf"}, {"region": {"page": 53}, "source_id": "mmlongbench:52b3137455e7ca4df65021a200aef724.pdf"}, {"region": {"page": 56}, "source_id": "mmlongbench:52b3137455e7ca4df65021a200aef724.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:32", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:32", "question": "How many distinct Netherlands location images does this slide use as examples?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "63", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:33", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:33", "question": "How many more millions of dollars was the median exit valuation in the USA compared to Europe at the time of the presentation?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Europe IPO", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:34", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:34", "question": "Which was greater at the time of the presentation, Europe IPO index value or US IPO index value?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "13", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 21}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 22}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 23}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 24}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 31}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:35", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:35", "question": "How many pages include charts whose horizontal-axis are set as year (like 2024)?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 33}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 37}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:36", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:36", "question": "How many human quotes are provided with source in this document?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['The Economist', 'TIME']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 33}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:37", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:37", "question": "Which two magazines' opinions are selected to illustrate the situation of German venture capital?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:38", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:38", "question": "Which newspaper's opinions are selected to illustrate the situation of Spain venture capital?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "10", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 23}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 24}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}, {"region": {"page": 30}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:39", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:39", "question": "In the slides, how many charts compare between ONLY US and Europe?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:40", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:40", "question": "In the slides, which chart compares between China and Japan?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "300 million", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 26}, "source_id": "mmlongbench:earlybird-110722143746-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:41", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:41", "question": "On the chart showing average exit cap, there is one point that US, Europe, and China have the same average exit cap value. What is this value in USD million? Give me an integer", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['OPTIMISATION', 'PREDICTIVE MODELING', 'FORECASTING', 'STATISTICAL ANALYSIS']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:42", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:42", "question": "According to the chart \"Levels of Analytics\", what are the four business analystics activities?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:43", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:43", "question": "According to the chart \"Levels of Analytics\", what are the four business cooperation activities?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Analysis", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:44", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:44", "question": "According to the flowchart \"Analytics Value Chain\", what comes between Reporting and Action?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Having lots of reports', 'Having lots of dashboards', 'Having lots of alerts', 'Having a hadopt cluster']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:45", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:45", "question": "What are the examples the slides show \"what does not make you data-driven\"", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Girl crash into lake following bad GPS direction", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 49}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 50}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:46", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:46", "question": "Which Youtube does the slides use to show the consequce of blindly following data?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:47", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:47", "question": "Which Youtube does the slides use to show the consequce of having a weak data leadership?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 51}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:48", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:48", "question": "How many hand drawn cartoons are included in the slides to illustrate the idea?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['a goals first approach', 'an objective, inquisitive culture', 'a visible, clear articulated strategy', 'strong data leadership', 'a strong testing culture', 'an open, sharing culture', 'a self-service culture', 'a broad data literacy']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 21}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 36}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 39}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 42}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 43}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 44}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}, {"region": {"page": 45}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:49", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:49", "question": "According to the slides, what are the things that a data-driven person has?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "highest paid person's opinion", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 25}, "source_id": "mmlongbench:ddoseattle-150627210357-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:50", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:50", "question": "In the cartoon in page 25, what does the hippo represent?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:51", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:51", "question": "In the cartoon in page 25, what does the elephant represent?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "51%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:52", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:52", "question": "Regarding the global iOS breakdown, what percentage of the pie chart was iOS 9?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "17.5", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:53", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:53", "question": "Between Java and WP, how large is the difference in percentage of their global developers mindshare?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Google Play store', '1.3']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:54", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:54", "question": "In 2014, which one has more mobile apps globally, Apple App store or Google Play store? How many Apps does it have in million?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "UC Browser Tiếng Việt", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}, {"region": {"page": 30}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:55", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:55", "question": "Which APP on APPOTA platform is top 10 Vietnam Android App, but not top 10 Vietnam iOS App?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:56", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:56", "question": "Which APP on APPOTA platform ranks 15th as Vietnam Android App?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "[1500, 8000]", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 26}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:57", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:57", "question": "In Q3 2015, what is the approximate range of cost in 1 day (dollar) of a mobile incentive platform in Vietnam for installing? Answer with a list of two number for boundary, for example [3000, 4000]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Appota", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 28}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}, {"region": {"page": 33}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:58", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:58", "question": "As of Q3 2015, are there more active Instagram users in Vietnam, or are there more fans for Appota's fan page? Directly answer 'Instagram' or 'Appota'.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['lower', '38']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:59", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:59", "question": "As of Q3 2015, is vietnam's adoption rate of iOS 9 higher or lower than a global average rate? What is the difference in percentage?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:60", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:60", "question": "As of Q3 2015, is vietnam's adoption rate of iOS 7 higher or lower than a global average rate? What is the difference in percentage?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Bluebird Award", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}, {"region": {"page": 31}, "source_id": "mmlongbench:reportq32015-151009093138-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:61", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:61", "question": "Which news appear in both Vietnam mobile news and APPOTA news?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['movement', 'birth', 'place of birth', 'member']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:2310.05634v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:62", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:62", "question": "In figure 1, which relation arrows do not point to specific leaf nodes?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "red", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:2310.05634v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:63", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:63", "question": "In figure 5, what is the color of the line that has no intersection with any other line?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:2310.05634v2.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:2310.05634v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:64", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:64", "question": "How many tables include \"F1\" as a metric?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.1", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:2310.05634v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:65", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:65", "question": "From the paper, which temperature gives ChatGPT the highest alignment score?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Evolutionary Question Generation", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:2310.05634v2.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:2310.05634v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:66", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:66", "question": "For dataset construction, which step takes the most word to describe than the others.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "If it can be partially verified by the knowledge graph G", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2310.05634v2.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:2310.05634v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:67", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:67", "question": "According to author's definition on conscious incompetence, when can a sentence map to both [NA] and a list of sub-graph knowledge?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:68", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:68", "question": "According to author's definition on conscious incompetence, when can a sentence map to either [NA] or a list of sub-graph knowledge?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "[16, 19, 25]", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:2401.18059v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:69", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:69", "question": "In figure 4, which nodes are retrieved by RAPTOR for both questions?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Context Length', 'F1']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:2401.18059v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:70", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:70", "question": "What are the horizontal and vertical axis of Figure 3 respectively?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Collapsed Tree Algorithm", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 18}, "source_id": "mmlongbench:2401.18059v1.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:2401.18059v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:71", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:71", "question": "Based on \"PSEUDOCODE FOR RETRIEVAL METHODS\", which algorithm has more number of lines? (Give the algorithm name)", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['green', 'yellow']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2401.18059v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:72", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:72", "question": "In Figure 1's demonstration, what are the color of the nodes that appear in more than one clusters?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:73", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:73", "question": "In Figure 1's demonstration, what are the color of the nodes that appear in all three clusters?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Gaussian Mixture Models', 'the high dimensionality of vector embeddings']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2401.18059v1.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:2401.18059v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:74", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:74", "question": "What model is the clustering algorithm of this paper based on, and what presents a challenge to it?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Slayer ← sorted(top k)[:k].nodes", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2401.18059v1.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:2401.18059v1.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:2401.18059v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:75", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:75", "question": "Write down the pseudo code from appendix that corresponds to step 5 of the tree traversal method", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Pre-training", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2312.10997v5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:76", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:76", "question": "In the figure that has a tree shape, what is the name of the branch that has the least leafs?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "9", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2312.10997v5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:77", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:77", "question": "In figure 3, how many distinct icons are used?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Copy is all you need.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:2312.10997v5.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:2312.10997v5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:78", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:78", "question": "What is the paper's full title that proposes the method that has a retrieval granularity of phrase?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:79", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:79", "question": "What is the paper that proposes the method that has a retrieval granularity of phrase and augmentation stage of tuning?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['POPQA', 'MS MARCO', 'SST-2']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:2312.10997v5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:80", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:80", "question": "According to table II, which are the datasets that has exactly three methods?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Iterative Retrieval ', 'Recursive Retrieval ', 'Adaptive Retrieval ']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:2312.10997v5.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:2312.10997v5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:81", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:81", "question": "Which subsection does the section \"AUGMENTATION PROCESS IN RAG\" include?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "PKG", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:2312.10997v5.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:2312.10997v5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:82", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:82", "question": "Which method integrates knowledge into white-box models via directive fine-tuning?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "10:08AM", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:honor_watch_gs_pro.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:honor_watch_gs_pro.pdf"}, {"region": {"page": 41}, "source_id": "mmlongbench:honor_watch_gs_pro.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:83", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:83", "question": "What is the time on the gallery screenshot when demostrating how to set galley watch faces?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "yellow", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 23}, "source_id": "mmlongbench:honor_watch_gs_pro.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:84", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:84", "question": "What is the color on the watch that shows the aerobic zone?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['gray', 'red']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:honor_watch_gs_pro.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:85", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:85", "question": "What are the colors of the icons that the users touch to move the app up and remove the app respectively?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:86", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:86", "question": "What is the color of the icons that the users touch to move the app down?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:honor_watch_gs_pro.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:honor_watch_gs_pro.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:87", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:87", "question": "How many distinct button functions are there during a workout?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "a quiet and relaxed environment", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:honor_watch_gs_pro.pdf"}, {"region": {"page": 21}, "source_id": "mmlongbench:honor_watch_gs_pro.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:88", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:88", "question": "If \"--\" is displayed as the resting heart rate reading, what kind of environment should the user stay in?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "54", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:nova_y70.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:nova_y70.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:89", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:89", "question": "How many distinct notification and status icons are displayed in the guidebook?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "set white balance", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 27}, "source_id": "mmlongbench:nova_y70.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:90", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:90", "question": "Under the pro mode to shoot, what is the function of the icon on right hand side of the icon that select a focus mode?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:91", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:91", "question": "Under the pro mode to shoot, what is the function of the icon on right hand side of the icon that set white balance?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Eiffel Tower", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:nova_y70.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:92", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:92", "question": "In the demostration of how to use a Knuckle to Take a Scrolling Screenshot, what buildings appear in the first picture?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['vision', 'bluetooth device']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:nova_y70.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:nova_y70.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:93", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:93", "question": "What devices other than phone are introduced for setting device+?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:watch_d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:94", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:94", "question": "How many incorrect postures of measuring blood pressure are demostrated if this guidebook?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2.5-3cm", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:watch_d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:95", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:95", "question": "At a position on your wrist that is about the width of two fingers away from the bottom of your palm, put the end of the ruler that comes with the watch through the buckle on the head. How many cm is this two fingers' distance according to the fogure?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:watch_d.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:watch_d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:96", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:96", "question": "How many steps are needed to customize the function of the Down Button?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Wake up the voice assistant.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:watch_d.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:97", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:97", "question": "What will happen when you press and hold the down button?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:98", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:98", "question": "What will happen when you press twice the down button?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "[\"Singapore-Cambridge GCE 'A' Level\", 'International Baccalaureate (IB) Diploma']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 25}, "source_id": "mmlongbench:2024.ug.eprospectus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:99", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:99", "question": "Which groups of applicants have the lastest end of application period according to this brochure?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Most Beautiful Campus", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2024.ug.eprospectus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:100", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:100", "question": "Among the top 10 reasons why NTU smart campus is for you, which is the reason that does not include any person in the corresponding figure?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Medicine", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:2024.ug.eprospectus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:101", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:101", "question": "Student from which major has a picture of playing badminton in this brochure?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:102", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:102", "question": "Student from which major has a picture of playing tennis in this brochure?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['College of Engineering', 'Nanyang Business School', 'National Institute of Education']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:2024.ug.eprospectus.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:2024.ug.eprospectus.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:2024.ug.eprospectus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:103", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:103", "question": "Which colleges/schools/institutes from NTU use year 3 students as cover page student model?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:2024.ug.eprospectus.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:2024.ug.eprospectus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:104", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:104", "question": "How many schools from College of Humanities, Arts, and Social Sciences are not individually introduced in detail?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "31", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:Bergen-Brochure-en-2022-23.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:Bergen-Brochure-en-2022-23.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:Bergen-Brochure-en-2022-23.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:Bergen-Brochure-en-2022-23.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:105", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:105", "question": "In chaper 2.2, how many museums in Bergen does this brochure introduce?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 23}, "source_id": "mmlongbench:Bergen-Brochure-en-2022-23.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:106", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:106", "question": "How many goblets appear in the figure showing Skyskraoeren?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:107", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:107", "question": "If I want to email to the museum in Bergen that shows Norweigian and international masterpieces, what email address should I send to?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "ESCAPE BRYGGEN", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:Bergen-Brochure-en-2022-23.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:Bergen-Brochure-en-2022-23.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:108", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:108", "question": "Which attraction in Bergen is not friendly to people with wheelchair?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "bus", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 32}, "source_id": "mmlongbench:Bergen-Brochure-en-2022-23.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:109", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:109", "question": "Which transport is included in \"Norway in a Nutshell\" but not \"Sognefjord in a Nutshell\"", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "19", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 46}, "source_id": "mmlongbench:Bergen-Brochure-en-2022-23.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:110", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:110", "question": "How many \"WC\"s are shown in the last page's map?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "300", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 27}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 39}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:111", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:111", "question": "Among all valid respondents,  how many rep/lean rep independents (the exact number, not the ratio) favor building a strong relationship with China on trade? Please give your answer to the nearest hundred, such as 100, 200 or 300....", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:112", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:112", "question": "Among all valid respondents,  how many hispanic independents (the exact number, not the ratio) favor building a strong relationship with China on trade? Please give your answer to the nearest hundred, such as 100, 200 or 300....", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 21}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 31}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:113", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:113", "question": "How many quotations from male respondent over 50 years old are included in this report?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 31}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}, {"region": {"page": 32}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:114", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:114", "question": "How many charts shown in this paper have results stratified by respondents' gender?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['China', 'North Korea', 'Russia', 'Iran']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:115", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:115", "question": "Please list all countries whose power and influence respondents would like to see diminished in the results of this report on Questions Q41a-t.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:PG_2021.03.04_US-Views-on-China_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:116", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:116", "question": "According to the survry,  what's the absolute percent change of dem/lean dem respondents viewing China's growing military power a very serious threat from 2020 to 2021?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:117", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:117", "question": "According to the survry,  what's the absolute percent change of respondents age above 65 viewing China's growing military power a very serious threat from 2020 to 2021?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "29%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:118", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:118", "question": "What's the percentage value of west Germany respondents viewing Germany's relationship with the United States is as important as its relationship with Russia?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['UK', 'Israel', 'China', 'Canada', 'Mexico', 'Japan', 'France', 'Austria', 'Russia']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:119", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:119", "question": "List all countries, except U.S. and German, that are mentioned in some illustrated surveys of this report. Note that we do not count EU as a single country.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "37%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:120", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:120", "question": "Among people who age 30-49, what is the difference of percentage value between Americen and Germen having positive view on their bilateral relationship?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Sharp divides in German and American views of security issues, from use of force to defense budgeting', 'Americans and Germans differ in their views of international organizations and leaders']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:121", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:121", "question": "Enumerate all chapter names (for example, Sharp divides in German and American views of security issues, from use of force to defense budgeting) that discussed the relationship between Germany or the U.S. with NATO. Your answer should be a list.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Republican support for increased defense spending from Europe has waned since 2017', 'Supporters of CDU/CSU more likely to favor increased defense spending', 'Democrats and Republicans are about as likely to name Germany as a top foreign policy partner, but Republicans are keener on Israel', 'Supporters of different parties take alternate stances on U.S.-German cooperation', 'Ideological differences in views of the UN, EU and Russia']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:122", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:122", "question": "List all titles of the charts (for example, Americans and Germans diverge sharply in their views of bilateral relations) in which the results are grouped by political affiliation.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "59%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:PG_2020.03.09_US-Germany_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:123", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:123", "question": "According to this report,  what percentage of Germany respondent view China or Japan as the top economic power? Give me a percantage value.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:124", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:124", "question": "According to this report,  what percentage of Chinese respondent view the U.S.  or Japan as the top economic power? Give me a percantage value.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:125", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:125", "question": "in which paragraph and page is the green italic numerals located?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "M.A.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:fd76bbefe469561966e5387aa709c482.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:126", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:126", "question": "What degree does LEBOUR have?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Diagram of Breccia Gashes with top denuded off but bottom shown in cliff", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:fd76bbefe469561966e5387aa709c482.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:127", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:127", "question": "What is the title of the of the Figure 2?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:128", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:128", "question": "What is the title of the of the Figure 7?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:fd76bbefe469561966e5387aa709c482.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:129", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:129", "question": "Among figure 1-4, how many figures show more than one breccia gash?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1884 April 5, 10 45 a.m.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:fd76bbefe469561966e5387aa709c482.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:130", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:130", "question": "When was the last serious shock recorded in the table? The answer needs to be accurate to the year, month, day, hour, and minute. For example: 1883 Dec.7, 2 8 p.m.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:131", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:131", "question": "what is the number of red logos in page 10?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "01983 873655", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:132", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:132", "question": "What is the telephone no for The Limes Residential Home?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:133", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:133", "question": "What is the telephone no for the Care Quality Commission", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Is the servife safe?', 'Is the service effective', 'Is the serve caring?', 'Is the service responsive?', 'Is the service well-led?']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:134", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:134", "question": "List the primary questions asked about the services in this report.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Page 1', 'Page 5']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:135", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:135", "question": "List all pages on which the logo of CQC locates. The answer should be formatted as a list like ['Page 2', 'Page 4'].", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:379f44022bb27aa53efd5d322c7b57bf.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:136", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:136", "question": "How many regulations of the HSCA 2008 are breached in all according to this report?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Allison Crimmins", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:137", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:137", "question": "Which individual lead this program?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:138", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:138", "question": "According to this document, Lebron Brennan belongs to which agency?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "9", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:139", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:139", "question": "How many chapters are in this report?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Climate Impacts on Water-Related Illness', 'Food Safety,Nutrition,and Distribution', 'Mental Health and Well-Being', 'Populations of  Concern']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:140", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:140", "question": "What are the last 4 chapters listed in the table of contents?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:141", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:141", "question": "How many chapter authors of this report come from Columbia University?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:142", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:142", "question": "How many pages include figures which have humans?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Chapter 2', 'Chapter 3', 'Chapter 5', 'Chapter 6']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:efd88e41c5f2606c57929cac6c1c0605.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:143", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:143", "question": "List the no. of chapters including research highlight. The answer shall be formatted as a list, for example, [\"Chapter 5\", \"Chapter 8\"].", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "11", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:149", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:149", "question": "Regarding the table for 3rd sampled question for the exam,  how many more times did Recruiter A view profiles than Recruiter B?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:150", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:150", "question": "Regarding the table for 3rd sampled question for the exam, how many more times did Recruiter A view profiles than Recruiter C?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "380", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:151", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:151", "question": "Regarding the table for 3rd sampled question for the exam, what is the total number of non-response InMails for Recruiter A and B?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:152", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:152", "question": "How many sampled questions illustrated in this slide?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Next-Gen Recruiter seems to eliminate the need for Boolean Search, will there still be questions about this?", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 24}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:153", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:153", "question": "What is the second FAQ shown in this slides?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:154", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:154", "question": "What is the fifth FAQ shown in this slides?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Melinda D.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:finalpresentationdeck-whatwhyhowofcertificationsocial-160324220748_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:155", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:155", "question": "The quote colored in blue is said by whom?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:156", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:156", "question": "The quote colored in green is said by whom?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:avalaunchpresentationsthatkickasteriskv3copy-150318114804-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:157", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:157", "question": "How many children are being read to in the photograph of Page 3?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "AVALAUNCH MEDIA", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:avalaunchpresentationsthatkickasteriskv3copy-150318114804-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:158", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:158", "question": "What is the name of the company that created this presentation?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Times New Roman', 'Arial', 'Comic Sans']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:avalaunchpresentationsthatkickasteriskv3copy-150318114804-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:159", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:159", "question": "According to the presentation, what are three bad fonts?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:avalaunchpresentationsthatkickasteriskv3copy-150318114804-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:160", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:160", "question": "How many reasons your presentation needs for asterisk kicked?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "44", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:161", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:161", "question": "How many districts were sampled during Wave III?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:162", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:162", "question": "How many districts were sampled during Wave IV?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "102", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:163", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:163", "question": "How many fieldwork personnel in all for Wave I and Wave II?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "yellow", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:164", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:164", "question": "What is the color of Kailali in the map of Page 12?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:165", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:165", "question": "What is the color of Beijing in the map of Page 12?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "27.2%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 30}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:166", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:166", "question": "Among 4021 respondents,  what is the percentage of them having a smart phone?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "14%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}, {"region": {"page": 41}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:167", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:167", "question": "How many percentage respondents in this survey access to internet few times a week or more?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:168", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:168", "question": "How many percentage respondents in this survey access to internet more than two times per month?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['Radio', 'Newspaper', 'Televison', 'Internet']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 36}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}, {"region": {"page": 37}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}, {"region": {"page": 39}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:169", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:169", "question": "Enumerate the media sources surveyed in this report.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1115", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:finalmediafindingspdf-141228031149-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:170", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:170", "question": "How many female respondents in wave III never listen to the radio in recent half year?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "29%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:171", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:171", "question": "In 2014, respondents who believe traditional values are outdated increased by how many percentage points compared to 2011?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:172", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:172", "question": "In 2014, respondents who believe traditional values are outdated increased by how many percentage points compared to 2001?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "29%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:173", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:173", "question": "In the year in which Palestine was added to the survey, respondents who believe traditional values are outdated increased by how many percentage points compared to 2011?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Oman", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:174", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:174", "question": "Which country had the highest percentage of respondents for whom traditional values mean a lot?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Arab youth are increasingly concerned about obesity and lifestyle diseases and do not believe that healthcare in their country is improving", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 33}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:175", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:175", "question": "What is the 8th (out of top10) findings listed in this presentation?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:176", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:176", "question": "What is the 11th findings listed in this presentation?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}, {"region": {"page": 37}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:177", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:177", "question": "Among the top 10 findings, how many of them are supported by statistical results grouped by the countries?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Egypt", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:asdaaburson-marstellerarabyouthsurvey2014-140407100615-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:178", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:178", "question": "According to this survey, which country’s youth show the greatest concern about unemployment problem?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "287", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:179", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:179", "question": "How many samples in MMMU belong to sociology subject?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.55%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:180", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:180", "question": "Among all samples belonging to 'Public Health' subject,  how many percentage samples showcased in this paper? Give me a ratio.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 21}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:181", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:181", "question": "How many Economics-related error cases fall into the Error Category of Perceptual Error?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Thermodynamics', 'Heat Transfer', 'Fluid Mechanics']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 111}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:182", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:182", "question": "Tell me all the subfields in Energy & Power subject for this dataset.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "12.1%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:183", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:183", "question": "According to this paper,  What's the GPT-4V's performance difference between CS subject and math subject on MMMU datasets?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:184", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:184", "question": "According to this paper,  What's the GPT-4V's performance difference between math subject and CS subject on MMLU datasets?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:185", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:185", "question": "According to this paper,  in which image type does gpt-4o demonstrate least proficiency compared with other image types? Choose one answer from this list: [\"Diagrams\", \"Tables\", \"Charts\", \"Chemical\", \"Photo\", \"Paintings\", \"Geometric\", \"Music\", \"Medical\"]", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "72.3%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:2303.08559v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:186", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:186", "question": "What is the performance of filter-then-rerank methods (w.o. ensemble) on 50-shot TACREV dataset?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:187", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:187", "question": "What is the performance of filter-then-rerank methods (w.o. ensemble) on 50-shot CONLL03 dataset?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Identify the entities expressed by each sentence, and locate each entity to words in the sentence. The possible entity types are: [Type_1], [Type_2], ..., [Type_N]. If you do not find any entity in this sentence, just output ‘Answer: No entities found.’", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:2303.08559v2.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:2303.08559v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:188", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:188", "question": "Repeat the instructions corresponding to the settings shown in red box of Figure 3 (left).", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "9", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 0}, "source_id": "mmlongbench:2303.08559v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:189", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:189", "question": "How many datasets are used for experiments of this paper in all? Give me an integer.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "The word {evt} triggers a MEET event: two or more entities come together at a single location and interact with one another face-to-face.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:2303.08559v2.pdf"}, {"region": {"page": 29}, "source_id": "mmlongbench:2303.08559v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:190", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:190", "question": "Tell me the prompt template used for label 'Contact.Meet' in filter-then-rerank method of this work.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:191", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:191", "question": "Tell me the prompt template used for label 'Contact.Meet' in purely LLM-based method of this work.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "0.5%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:2303.08559v2.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:2303.08559v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:192", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:192", "question": "If I drop the green module shown in Figure 6,  what is the absolute percentage drop on 100-shot TACREV setting?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Go to the medical department you registered at (i.e. internal medicine, surgical medicine, dental medicine)", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 27}, "source_id": "mmlongbench:guojixueshengshenghuozhinanyingwen9.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:193", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:193", "question": "I want to see a doctor in the campus hospital.  After registering at the registration area, what is the next step?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:guojixueshengshenghuozhinanyingwen9.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:194", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:194", "question": "How many figures in this document show the old gate of Tsinghua (\"Er Xiao Men\" in Chinese)?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:195", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:195", "question": "According to this document,  what is the main color of Tsinghua Campus Bicycle? Give me one color only.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "C", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 34}, "source_id": "mmlongbench:guojixueshengshenghuozhinanyingwen9.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:196", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:196", "question": "What is the place locating at the intersection between Zijing Road and Xuetang Road? Give me the letter shown in the campus map of this guideline.  For example, \"A\".", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 23}, "source_id": "mmlongbench:guojixueshengshenghuozhinanyingwen9.1.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:guojixueshengshenghuozhinanyingwen9.1.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:guojixueshengshenghuozhinanyingwen9.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:197", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:197", "question": "How many QR codes are shown in the \"Academics and Related Resources\" part of this guidebook?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Ten Years After Café", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:guojixueshengshenghuozhinanyingwen9.1.pdf"}, {"region": {"page": 34}, "source_id": "mmlongbench:guojixueshengshenghuozhinanyingwen9.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:198", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:198", "question": "I'm at the location \"J\" shown in the campus map. Tell me the name of the nearest coffee shop.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Page 45', 'Page 49', 'Page 50']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 45}, "source_id": "mmlongbench:Macbook_air.pdf"}, {"region": {"page": 49}, "source_id": "mmlongbench:Macbook_air.pdf"}, {"region": {"page": 50}, "source_id": "mmlongbench:Macbook_air.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:199", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:199", "question": "Tell me all the pages introducing how to reinstall the software. Your answer should be formatted as a list about \"Page X\",  for example,  [\"Page 17\", \"Page 25\"]", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 33}, "source_id": "mmlongbench:Macbook_air.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:200", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:200", "question": "How many figures are shown in this guidebook to teach users \"Two-finger scrolling\" tips?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Headphone jack', 'USB 2.0 port', 'Micro-DVI port']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 30}, "source_id": "mmlongbench:Macbook_air.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:201", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:201", "question": "According to this guideline, enumerate all jacks or ports located at the right side of MacBook Air.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Canada', 'Europe', 'Korea', 'Singapore', 'Taiwan', 'China', 'Netherlands', 'Germany']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 66}, "source_id": "mmlongbench:Macbook_air.pdf"}, {"region": {"page": 67}, "source_id": "mmlongbench:Macbook_air.pdf"}, {"region": {"page": 68}, "source_id": "mmlongbench:Macbook_air.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:202", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:202", "question": "List all the countries/regions mentioned in the \"Regulatory Compliance Information\" part of this guidebook (except the U.S.). Your answer should be in English.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1-800-275-2273", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 54}, "source_id": "mmlongbench:Macbook_air.pdf"}, {"region": {"page": 55}, "source_id": "mmlongbench:Macbook_air.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:203", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:203", "question": "I'm a Macbook Air user in Mexico.  According to this guidebook, which number shall I call for seeking Applecare service and support?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Ambient light sensor", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 26}, "source_id": "mmlongbench:Macbook_air.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:204", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:204", "question": "According to this guideline, what is on the left side of the MacBook Air's camera?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:205", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:205", "question": "According to this guideline, what is on the left side of the Thinkpad's camera?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "541", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:206", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:206", "question": "Among the adults conducted the survey on April 25 - May 1 2018, how many adults rated Trump's government ethical standards as poor?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:207", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:207", "question": "Among the adults conducted the survey on April 25 - May 1 2022, how many adults rated Trump's government ethical standards as poor?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Make good decisions about economic policy ', 'Make wise decisions about immigration policy ']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:208", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:208", "question": "According to the survey on April 25 - May 1, 2018, what are the domains with the highest percentage that adults are very confident and not at all confident of Donald Trump's govenment? Please write the answer in the list format and with alphabetical order, e.g., [\"A\",\"B\"]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:209", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:209", "question": "According to the survey on April 25 - May 1, 2018, what are the domains with the highest percentage that adults are very confident and not at all confident of  Joe Biden's govenment? Please write the answer in the list format and with alphabetical order, e.g., [\"A\",\"B\"]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "21%", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:210", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:210", "question": "What percentage of the party that holds the highest total percentage of good policy ideas and high ethical standards reported that they followed the news about congressional elections in their state very closely in the survey of U.S. adults conducted from April 25 to May 1, 2018?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:211", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:211", "question": "What percentage of the party that holds the highest total percentage of good policy ideas and high ethical standards reported that they followed the news about congressional elections in their state very closely in the survey of U.S. adults conducted from April 25 to May 1, 2022?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "92%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:212", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:212", "question": "What is the percentage of registered voters who support or lean toward the candidate from the party with the higher total percentage of good policy ideas and high ethical standards and closely follow congressional elections in their district in the survey of U.S. adults conducted April 25 - May 1, 2018?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:213", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:213", "question": "What is the percentage of registered voters who support or lean toward the candidate from the party with the higher total percentage of good policy ideas and high ethical standards and closely follow congressional elections in their district in the survey of Chinese adults conducted April 25 - May 1, 2018?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "7.0", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:214", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:214", "question": "What is the percentage gap between the youngest age group and the oldest age group that strongly disapproves of the way the president, who started his job in 2018, was handling his job in the survey of U.S. adults conducted April 25-May 1, 2018? Please write the answer in float format", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:215", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:215", "question": "What is the percentage gap between the youngest age group and the oldest age group that strongly disapproves of the way the president, who started his job in 2018, was handling his job in the survey of Chinese adults conducted April 25-May 1, 2018?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "128", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:05-03-18-political-release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:216", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:216", "question": "How many Demoncratic people in the survey of U.S. adults conducted April 25- May 1, 2019 said neither the Republican Party nor the Democratic Party  has ‘high ethical standards'?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:217", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:217", "question": "How many non-partisan people in the survey  of U.S. adults conducted April 25- May 1, 2019 said neither the Republican Party nor the Democratic Party  has ‘high ethical standards'?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "41", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:218", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:218", "question": "What is the percentage of people who do not go online?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "4087", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:219", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:219", "question": "How many people who do not go online or only use SNS in the Older Adults and Technology by Princeton Survey Research Associates International from July 18 to September 30, 2013?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:220", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:220", "question": "How many people who do not go online or only use SNS in the Older Adults and Technology by Princeton Survey Research Associates International from July 18 to September 30, 2022 ?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "12%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:221", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:221", "question": "What is the percentage gap between male 65+ age group who use internet and  broadband at home in the Pew Research Center’s Internet Project July 18-September 30, 2013 tracking survey? Please write the answer into float format.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:222", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:222", "question": "What is the gap between the percentage of 65+ age group who go online in 2000 and the percentage of 80+ age group still go online on 2022?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['90%', '42%', '39%']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:223", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:223", "question": "Among the Higher-income seniors, what are the percentage of them go online, has smartphone phone, and own a tablet computer? Please write the answer in the list format and in descend order,e.g., [\"9%\",\"8%\"] in the Pew Research Center’s Internet Project July 18-September 30, 2013 tracking survey?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:224", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:224", "question": "Among the Higher-income seniors, what are the percentage of them go online, has smartphone phone, and own a tablet computer? Please write the answer in the list format and in descend order,e.g., [\"9%\",\"8%\"] in the Pew Research Center’s Internet Project July 18-September 30, 2022 tracking survey?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "1251", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:225", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:225", "question": "How many 65+ age group people go online 3-5 times per week or Every day in the Pew Research Center’s Internet Project July 18-September 30, 2013 tracking survey?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:226", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:226", "question": "How many 65+ age group people go online 3-5 times per week or Every day in the Pew Research Center’s Internet Project July 18-September 30, 2020 tracking survey?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "301", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 18}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:227", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:227", "question": "What is the gap of 65+ people with College graduate contain a cell phone and a tablet computer in the Pew Research Center’s Internet Project July 18-September 30, 2013 tracking survey. ?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:228", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:228", "question": "What is the gap of infants with College graduate contain a cell phone and a tablet computer in the Pew Research Center’s Internet Project July 18-September 30, 2013 tracking survey?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "73.0", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:PIP_Seniors-and-Tech-Use_040314.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:229", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:229", "question": "What is the gap between male 65+ age group who use internet and  broadband at home in the Pew Research Center’s Internet Project July 18-September 30, 2013 tracking survey? Please write the answer into float format.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Capt. John W. Sanders", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:0b85477387a9d0cc33fca0f4becaa0e5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:230", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:230", "question": "Who is the commanding officer in the first figure on the second page?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:231", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:231", "question": "Who is the commanding officer at the last page?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Tim Ziemer", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:0b85477387a9d0cc33fca0f4becaa0e5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:232", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:232", "question": "Who Visited the U.S. Naval Medical Research centre on November 29, 2012 in the document?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:233", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:233", "question": "Who Visited the U.S. Naval Medical Research centre on November 29, 2020?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Walter Gwenigale', 'Fatorma Bolay', 'Vernon Graham']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:0b85477387a9d0cc33fca0f4becaa0e5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:234", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:234", "question": "With whom did the NAMRU-3 team visit Monrovia, Liberia, in November 2012? Enumerate their names within a list.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:0b85477387a9d0cc33fca0f4becaa0e5.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:0b85477387a9d0cc33fca0f4becaa0e5.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:0b85477387a9d0cc33fca0f4becaa0e5.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:0b85477387a9d0cc33fca0f4becaa0e5.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:0b85477387a9d0cc33fca0f4becaa0e5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:235", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:235", "question": "Among all the pictures, how many of them contain only one person?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "9", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:0b85477387a9d0cc33fca0f4becaa0e5.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:236", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:236", "question": "Among all the pictures, in which page the picture contains the most number of people? Give me the page no. as an integer.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2009-07", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:237", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:237", "question": "What date is mentioned on the second page? Format the date as YYYY-MM", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['combshj@unk.edu']", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:238", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:238", "question": "What is the Email address in this document on the second cover page?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "538", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:239", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:239", "question": "How many square miles did the Hamilton country covers on year 1882? Return me a rounded integer.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1862", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:240", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:240", "question": "What was the population of the city with the largest font on the map on Page 3 in 1890? Answer in int format", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['p4', 'p7', 'p9']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:241", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:241", "question": "How many pages does websites address appeared? List all the pages in list format, for example [\"p1\",\"p2\"]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "18.40%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:242", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:242", "question": "What percentage of land area was rezoned in the Bronx from 2003-2007?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "435000000", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:243", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:243", "question": "What is the residential capacity of Staten Island from 2003 to 2007? Give me an integer.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "The impact of rezonings", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:244", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:244", "question": "What is the first subdivision topic in the title of the Areas for future research?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:245", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:245", "question": "What is the color of the bike in the picture on page 2?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:246", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:246", "question": "Which area of New York has more than 23% land area rezoned?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:247", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:247", "question": "Which area of New York has more than 23% land area rezoned on 2001?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:248", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:248", "question": "What is the sum of the number of cars in the figures on page 2 and the number of bars in the figures on page 4? Write the answer in int format.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['1', '2', '5']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:249", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:249", "question": "List the number of the people in the figure in page 6, the number of buildings in page 14,  and the number of legends in figure A in page 3. Write the answer in the list format, for example, [\"1\",\"2\"]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:8e7c4cb542ad160f80fb3d795ada35d8.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:250", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:250", "question": "How many research questions does this paper answer?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "198", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:251", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:251", "question": "On the slide that starts with the text Country Overview in the top-left corner and is followed with a bulleted list with the first entry in that list being Population: (July 2013 est.) 251,160,124, how much greater is the GDP per capita for 2012 than 2011?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:252", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:252", "question": "On the slide that starts with the text Country Overview in the top-left corner and is followed with a bulleted list with the first entry in that list being Population: (July 2013 est.) 251,160,124, how much greater is the GDP per capita for 2020 than 2019?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "26%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:253", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:253", "question": "Looking at the Slide of country overview, by what percent did \"Smartphone Penetration\" grow between 2013 and 2015?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:254", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:254", "question": "Looking at the Slide of country overview, by what percent did \"Bike Penetration\" grow between 2013 and 2015?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Christians", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:255", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:255", "question": "Which group accounts for the second largest share in terms of religious demographics in Slide 4 in 2014?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:256", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:256", "question": "Which group accounts for the second largest share in terms of religious demographics in Slide 4 in 2018?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "XL", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:257", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:257", "question": "What is the name of the second largest telecom operator in terms of the number of subscribers in the period 2013-2014?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['3', 'Esia', 'Smartfren']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:258", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:258", "question": "Comparing to the Telecom Operators in 2014 and 2013-2014, What are the Operators that are not in common? Please list the names in list format, e.g., [\"a\",\"b\"]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['XL', 'Kotakgame.com', 'GameQQ.net']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:259", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:259", "question": "What are the Online Games native major internet companies and the Telecom Operator name of the second largest Prepaid ARPU in 2008? Please list the answer in list with reverse alphabetical order.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 34}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:260", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:260", "question": "In the slide that includes Mobile Internet Demographics and Trends, how many colors are there in the chart in the top right corner?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:261", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:261", "question": "In the slide that includes Mobile Ads Network (2), how many colors are there in the chart in the top right corner?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "0-20", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 31}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:262", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:262", "question": "What is the chart legend name that with a flag in the slide 31 have from 2008-2012?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:263", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:263", "question": "How many data did the country that with a flag in the slide 31 have from 2022?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "24", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 18}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 21}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 23}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:indonesiamobilemarketresearch-ag-150106055934-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:264", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:264", "question": "How many website URLs are included in the Section Internet Industry in the slides?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:caltraincapacitymountainview1-150701205750-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:265", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:265", "question": "How many lines go through the Union Square/Market Street station in San Francisco in the slides?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:266", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:266", "question": "How many lines go through the Union Square/Market Street station in Toronto in the slides?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:caltraincapacitymountainview1-150701205750-lva1-app6891_95.pdf"}, {"region": {"page": 21}, "source_id": "mmlongbench:caltraincapacitymountainview1-150701205750-lva1-app6891_95.pdf"}, {"region": {"page": 23}, "source_id": "mmlongbench:caltraincapacitymountainview1-150701205750-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:267", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:267", "question": "How many figures of trains are appeared in the documents?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:268", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:268", "question": "How many figures of airplanes are appeared in the documents?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:caltraincapacitymountainview1-150701205750-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:269", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:269", "question": "How many colors are in the slide 14?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:caltraincapacitymountainview1-150701205750-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:270", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:270", "question": "In the figure on slide 11, how many more locations are there for Established compared to Developing?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:271", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:271", "question": "In the figure on slide 14, how many more locations are there for Established compared to Developing?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "Increased or sustained", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:digitalmeasurementframework22feb2011v6novideo-110221233835-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:272", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:272", "question": "In 24 months after the date of the presentation, what is expected to happen to the value of data visualization?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:273", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:273", "question": "In 48 months after the date of the presentation, what is expected to happen to the value of data visualization?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Customer Focused", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:digitalmeasurementframework22feb2011v6novideo-110221233835-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:274", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:274", "question": "Did more respondents say they are Product/Brand Focused or Customer Focused?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "[2, 2]", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:digitalmeasurementframework22feb2011v6novideo-110221233835-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:275", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:275", "question": "How many shapes and colors of the shapes in the Slide 9? Please write the answer in the list format, e.g., [1,2]", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:276", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:276", "question": "How many shapes and colors of the shapes in the Slide 5? Please write the answer in the list format, e.g., [\"1\",\"2\"]", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "3%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:digitalmeasurementframework22feb2011v6novideo-110221233835-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:277", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:277", "question": "What is the subtraction result of ability to get the data's percentage and concerns with the data's percentage in slide 11?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 95}, "source_id": "mmlongbench:digitalmeasurementframework22feb2011v6novideo-110221233835-phpapp01_95.pdf"}, {"region": {"page": 107}, "source_id": "mmlongbench:digitalmeasurementframework22feb2011v6novideo-110221233835-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:278", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:278", "question": "What is the sum of the total number of paid search's conversions in the year of 2007, 2008 and the number of green bars in the heroes happen here launch?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:279", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:279", "question": "What is the sum of the total number of paid search's conversions in the year of 2007, 2008, 2009 and the number of green bars in the heroes happen here launch?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['black', 'white']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 133}, "source_id": "mmlongbench:digitalmeasurementframework22feb2011v6novideo-110221233835-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:280", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:280", "question": "In the Slide that mentioned Qualitative vs Quantitative Measurement, what are the colors of the text \"Qualitative\" and the background color of it? Please list the colors in list with alphabetical order, e.g., [\"black\",\"red\"]", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:2310.07609v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:281", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:281", "question": "How many numbers with blue color are mentioned in the example figure of question-guidue reasoning?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:282", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:282", "question": "How many numbers with purple color are mentioned in the example figure of question-guidue reasoning?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2310.07609v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:283", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:283", "question": "In the proposed system, how many modules are implemented with Large Language Models backbone?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:284", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:284", "question": "In the proposed system, how many modules are implemented with BERT's backbone?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "4. The final prediction result with rationale.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:2310.07609v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:285", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:285", "question": "In the figure that locates at the top of page 5, what is the final step? Please write down the answer in string format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Yellow", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2310.07609v1.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:2310.07609v1.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:2310.07609v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:286", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:286", "question": "What is the color of the model rectangle in the figure of page 4 that appears both in QA model and Reasone moduler in the paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2310.07609v1.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:2310.07609v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:287", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:287", "question": "How many prompts instruction examples are listed in the main body of the paper? Please writhe down the answer in float format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.45", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:2310.07609v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:288", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:288", "question": "What is the average performance gap between the ProgramFC performance and our proposed system in the HOVER dataset? Please write down the answer in float format with 2 decimals.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8980", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2303.05039v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:289", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:289", "question": "How many items in total of Amazon related datasets in the paper? Please write the answer in int format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:290", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:290", "question": "How many items in total of Shopee related datasets in the paper? Please write the answer in int format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "466.43", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2303.05039v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:291", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:291", "question": "What is the average number of words per user used for the experiments in the dataset from which a data sample of a user's Receptiviti score is shown? Please write the answer in float format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:292", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:292", "question": "What is the average number of words per user used for the experiments in the dataset from which a data sample of a user's LIWC score is shown? Please write the answer in float format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Orange", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2303.05039v2.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:2303.05039v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:293", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:293", "question": "What is the color of the personality vector in the soft-labled  personality embedding matrix that with the highest Receptiviti score for User A2GBIFL43U1LKJ?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['71.02', '75.38']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:2303.05039v2.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:2303.05039v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:294", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:294", "question": "What are the personality scores of the data sample with extreme personality cases, in which the personality vector color is blue in the overall model structure? Please write down the answer in list format with ascending order, e.g., [\"1\",\"2\"]", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.833", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:2303.05039v2.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:2303.05039v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:295", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:295", "question": "What is the highest hit rate number in the dataset for the personality located furthest to the left in the distribution of personality traits figure?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "H@10", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:2303.05039v2.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:2303.05039v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:296", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:296", "question": "What is the evaluation metric that has highest number for the method located at the bottom of the model structure figure across the three datasets? Please write down the answer with string format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['3', '1']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2305.13186v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:297", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:297", "question": "How many green and grey rectangles in the first figure of the paper? Please list the numbers in the list format by descending order. e.g., [\"2\",\"1\"]", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:298", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:298", "question": "How many green and grey triangles in the first figure of the paper? Please list the numbers in the list format by descending order. e.g., [\"2\",\"1\"]", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2305.13186v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:299", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:299", "question": "How many reasoning steps are involved in the figure 1 in the paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "197940", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2305.13186v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:300", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:300", "question": "How many more claims does the Wiki Table datasets have comparing to scientific articles datasets? Please writeh the answer in int format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:301", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:301", "question": "How many more claims does the Health Table datasets have comparing to Newspaper datasets? Please writeh the answer in int format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "453.25", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2305.13186v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:302", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:302", "question": "How many USD dollars will the author spend if the annotators finish annotating all of claims of the authors' proposed dataset? Please write the answer in float format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "245", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2305.13186v3.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:2305.13186v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:303", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:303", "question": "How many claims are with the highest percentage of reasoning steps  in the author's proposed dataset?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "10", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:2305.13186v3.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:2305.13186v3.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:2305.13186v3.pdf"}, {"region": {"page": 22}, "source_id": "mmlongbench:2305.13186v3.pdf"}, {"region": {"page": 23}, "source_id": "mmlongbench:2305.13186v3.pdf"}, {"region": {"page": 24}, "source_id": "mmlongbench:2305.13186v3.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:2305.13186v3.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:2305.13186v3.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:2305.13186v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:304", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:304", "question": "How many figures are there in the Appendix in the paper? Please write down the answer into int format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2.92", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 23}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:305", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:305", "question": "A transmitter operating at a frequency of 500 MHz has a rated maximum output power of 10 W. Using the recommended separation distances between portable and mobile RF communications equipment and the M270TF-XXX / M320TF-XXX, calculate the minimum separation distance required for this transmitter. Write down the answer in float format with 2 decimals.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:306", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:306", "question": "A transmitter operating at a frequency of 500 MHz has a rated maximum output power of 10 W. Using the recommended separation distances between portable and mobile RF communications equipment and the M250, calculate the minimum separation distance required for this transmitter. Write down the answer in float format with 2 decimals.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Operate the unit on 100-240V AC only.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 25}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:307", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:307", "question": "For the users with the safety approval of UL Listed and CSA , what is the second bullet point for safety?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:308", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:308", "question": "For the users with the safety approval of UL Listed and CSA , what is the tenth bullet point for safety?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "3.6.8  Flat surface for better maintenance", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 24}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:309", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:309", "question": "In the Section Warning and Cautions, what is the subsection name contains a figure of screen in the guidebook?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:310", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:310", "question": "In the Section Introduction, what is the subsection name contains a figure of screen in the guidebook?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "\"3840 x 2160\" at 30 Hz", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:311", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:311", "question": "In the table at page 28,  which signal has the least frequency in the guidebook?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 31}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}, {"region": {"page": 32}, "source_id": "mmlongbench:91521110100M_4K_UHD_Display_User_Manual_V1.1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:312", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:312", "question": "Among the tables in Appendix, how many columns does table has the largest layout contain? Write the answer in int format.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:bdf54dxa.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:313", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:313", "question": "Which step in the figure at the top of page 10 poured the salt to the dishwasher? Write the answer in int format.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:314", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:314", "question": "Which step in the figure at the top of page 10 poured the salt to the microwave? Write the answer in int format.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:bdf54dxa.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:315", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:315", "question": "How many figures in page 13 contains the left arrows? Write down the number in float format with 2 decimals.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:316", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:316", "question": "How many figures in page 13 contains the up arrows? Write down the number in float format with 2 decimals.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:bdf54dxa.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:bdf54dxa.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:317", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:317", "question": "In the adjusting the upper basket step, how many up arrows are contained in the figures? Write down the answers in float format with 1 decimal.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "100", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 24}, "source_id": "mmlongbench:bdf54dxa.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:318", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:318", "question": "To built-in install for the integrated diswasher model,  how many mm should the diswasher leave between the cabinet bottom and floor?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "17", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 30}, "source_id": "mmlongbench:bdf54dxa.pdf"}, {"region": {"page": 31}, "source_id": "mmlongbench:bdf54dxa.pdf"}, {"region": {"page": 32}, "source_id": "mmlongbench:bdf54dxa.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:319", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:319", "question": "How many possible problems does the diswasher may encounter before calling for service?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:obs-productdesc-en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:320", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:320", "question": "How many \"objects\" with black color does the figure in Page 2 contains?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:321", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:321", "question": "How many \"subjects\" with black color does the figure in Page 2 contains?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Data restore', 'Designed durability', 'Designed durability (multi-AZ)']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:obs-productdesc-en.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:obs-productdesc-en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:322", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:322", "question": "What are the compared item with blue color in Table1-1? Write the answer in list format with alphabetical order.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:323", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:323", "question": "What are the compared item with red color in Table1-1? Write the answer in list format with alphabetical order.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Multi-AZ storage", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:obs-productdesc-en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:324", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:324", "question": "What is the benefit of level-2 in the system has passed TRUCS in Page 8?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:325", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:325", "question": "What is the benefit of level-6 in the system has passed TRUCS in Page 8?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:obs-productdesc-en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:326", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:326", "question": "How many end-use mobile electronic devices are contained in the Enterprise cloud boxes (web disks) in the flow chart?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "42", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 41}, "source_id": "mmlongbench:obs-productdesc-en.pdf"}, {"region": {"page": 42}, "source_id": "mmlongbench:obs-productdesc-en.pdf"}, {"region": {"page": 43}, "source_id": "mmlongbench:obs-productdesc-en.pdf"}, {"region": {"page": 44}, "source_id": "mmlongbench:obs-productdesc-en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:327", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:327", "question": "How many common operations supported by each system-defined policy or role of OBS? Write down the answer in int format.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:GPL-Graduate-Studies-Professional-Learning-Brochure-Jul-2021.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:328", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:328", "question": "How many people with scarf are there in  Page 5?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:329", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:329", "question": "How many people with sun glassess are there in  Page 5?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['3', '4']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:GPL-Graduate-Studies-Professional-Learning-Brochure-Jul-2021.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:330", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:330", "question": "How many male and female are there in page 17? Please write the answer in list format, e.g., [\"1\",\"2\"]", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['0', '0']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:GPL-Graduate-Studies-Professional-Learning-Brochure-Jul-2021.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:331", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:331", "question": "How many dogs and cats are there in page 17? Please write the answer in list format, e.g., [\"1\",\"2\"]", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:GPL-Graduate-Studies-Professional-Learning-Brochure-Jul-2021.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:GPL-Graduate-Studies-Professional-Learning-Brochure-Jul-2021.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:332", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:332", "question": "In the Leadership program section, how many male are there with glasses? Please write the answer in floqt format.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:GPL-Graduate-Studies-Professional-Learning-Brochure-Jul-2021.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:333", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:333", "question": "In the figure at Page at the second last, howm many people are standing?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Leaders in Education Programme", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:GPL-Graduate-Studies-Professional-Learning-Brochure-Jul-2021.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:334", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:334", "question": "Which program has the longest time period among all the leadership programmes?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['MA (Humanities Education)', 'MSc (Exercise & Sport Studies)', 'MSc (Life Science)', 'MSc (Mathematics for Education)']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:GPL-Graduate-Studies-Professional-Learning-Brochure-Jul-2021.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:335", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:335", "question": "Which programme by coursework with disciplinary content allows to have the maximum of 3 years full-time duration? Write down the answer with alphabetical order.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:welcome-to-nus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:336", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:336", "question": "How many people with red shirt are there in the Page 14?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:337", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:337", "question": "How many people with white shirt are there in the Page 14?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['15', '9']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:welcome-to-nus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:338", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:338", "question": "How many NUSSU constituent clubs and NUSSU committees are there respectively? Please write the answer in list format.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:339", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:339", "question": "How many NTUSU constituent clubs and NTUSU committees are there respectively? Please write the answer in list format.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:welcome-to-nus.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:welcome-to-nus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:340", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:340", "question": "How many people are there in the Figure from page 19 to page 21? Please write the answer in float format.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['2', '5', '7']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:welcome-to-nus.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:welcome-to-nus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:341", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:341", "question": "How many Halls of Residence,  Residential colleges,  and student residences are there in NUS respectively? Please write the answer in list with ascending order, e.g., [\"1\",\"2\"]", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Computing Commons', 'Central Library', 'Medical Library', 'Chinese Library', 'Science Library', 'Hon Sui Sen Memorial Library']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:welcome-to-nus.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:welcome-to-nus.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:342", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:342", "question": "For the learning resources, which facilities are opened on Saturday in NUS Kent Ridge Campus? Give me a list.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "26%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:343", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:343", "question": "What is the proportion of people who say they are very confident in Trump's ability to handle an international crisis?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:344", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:344", "question": "What is the proportion of people who say they are very confident in Lincoln's ability to handle an international crisis?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "16%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:345", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:345", "question": "What is the percentage difference between the proportion of people who are very confident in Trump's ability to make wise decisions about immigration policy and those who are not confident at all?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:346", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:346", "question": "What is the percentage difference between the proportion of people who are very confident in Trump's ability to make wise decisions about healthy policy and those who are not confident at all?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "10%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:347", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:347", "question": "How many Republicans are not confident at all that Trump keeps his own business interests separate from the decisions he makes as president?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "49%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:348", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:348", "question": "In the year when 58% of people thought it was too early to tell if Trump was a successful president, how many people believed that his economic policies had not much effect on the economic situation?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:349", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:349", "question": "In the year when 40% of people thought it was too early to tell if Trump was a successful president, how many people believed that his economic policies had not much effect on the economic situation?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "45%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:350", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:350", "question": "Compared to Trump in January 2019, what higher percentage of Republicans believed that Clinton in February 1995 would be an unsuccessful president in the long run?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "10%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:351", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:351", "question": "In 2019, what is the percentage difference between those who are not too confident that Robert Mueller is conducting a fair investigation into Russian involvement in the 2016 election and those who were very confident in January 2018?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "37%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}, {"region": {"page": 21}, "source_id": "mmlongbench:PP_2019.01.17_Trump-economy_FINAL2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:352", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:352", "question": "In 2019, by what percentage is the total proportion of people who say they expect their personal financial situation to improve some or a lot over the course of the next year higher than the proportion of those saying jobs are difficult to find in their community?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "19%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:353", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:353", "question": "How many EU people believe that they will have more influence in world affairs after the coronavirus outbreak compared to before the outbreak?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:354", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:354", "question": "How many African people believe that they will have more influence in world affairs after the coronavirus outbreak compared to before the outbreak?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Liberal", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:355", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:355", "question": "In the United States, which group has the highest proportion of people who believe that the U.S. will have less influence in world affairs after the coronavirus outbreak compared to before the outbreak?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:356", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:356", "question": "In the Russian, which group has the highest proportion of people who believe that the U.S. will have less influence in world affairs after the coronavirus outbreak compared to before the outbreak?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "53%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:357", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:357", "question": "How many Hispanics believe the U.S. can learn a great deal from other countries around the world about ways to slow the spread of coronavirus?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:358", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:358", "question": "How many Frenchman believe the U.S. can learn a great deal from other countries around the world about ways to slow the spread of coronavirus?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "13%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:359", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:359", "question": "What is the percentage difference between the proportion of people who believe the U.S. should help other countries deal with their problems and those who believe the U.S. has done a poor job in dealing with the coronavirus outbreak?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "35%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:360", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:360", "question": "How much greater is the partisan divide in the U.S. compared to the U.K. regarding whether the handling of the coronavirus outbreak has been good/excellent?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:361", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:361", "question": "How much greater is the partisan divide in the Russian compared to the Japan regarding whether the handling of the coronavirus outbreak has been good/excellent?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "1%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:362", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:362", "question": "In the United States, by what percentage is the proportion of people who believe the World Health Organization has done a good job in dealing with the coronavirus outbreak lower than the proportion of people who believe the U.S. has done a good job in dealing with the coronavirus outbreak?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "29%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:PG_2020.05.21_International-Cooperation-COVID_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:363", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:363", "question": "How many more people over 65 years old do not trust information from the World Health Organization compared to postgraduates who do not trust information from the European Union in regard to the coronavirus outbreak?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['State Bank of India', 'Bank of Baroda', 'Bank of India', 'Citibank N.A.', 'State Bank of Hyderabad', 'State Bank of Travancore', 'The Hongkong and Shanghai Banking Corporation', 'Unioon Bank of India']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:364", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:364", "question": "What are the bankers' names associated with GODFREY PHILLIPS INDIA LIMITED?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:365", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:365", "question": "What are the bankers' names associated with Apple Inc?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "2.4%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:366", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:366", "question": "What was the value of absolute percentage shortfall in India's GDP growth from 2002 to 2003 compared to the forecast?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:367", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:367", "question": "What was the percentage shortfall in Japan's GDP growth from 2002 to 2023 compared to the forecast?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['V.K.K. MENON ROAD (SAHAR ROAD),CHAKALA,ANDHERI (EAST),MUMBAI-400099', 'International Tobacco Company Ltd. Delhi- Meerut Road, Guldhar, Ghaziabad']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:368", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:368", "question": "Where are the two plants of GODFREY PHILLIPS INDIA LIMITED located?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:369", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:369", "question": "According this document, where are the two plants of Apple Inc. located?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Mr. S.V. Shanbhag", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:370", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:370", "question": "Which executive directors have attended Board Meetings more than 6 times?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "30", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:371", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:371", "question": "How many companies do Mr. R.A. Shah, Mr. S.V. Shanbhag, and Mr. C.M. Maniar collectively hold directorships of the Board?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Mr.R.A.Shah', 'Mr.Lalit Bhasin', 'Mr.Anup N.Kothari', 'Mr.C.M.Maniar', 'Mr.O.P.Vaish']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:372", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:372", "question": "Who are the non-executive and independent directors of GODFREY PHILLIPS INDIA LIMITED? Enumerate all of them in a list.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "155.98", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:373", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:373", "question": "How much higher was the proposed dividend paid (Rupees in lacs) in 2002 compared to 2001?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:374", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:374", "question": "How much higher was the dividend paid in 2003 compared to 2002?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['5.3%', '5.2%']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:375", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:375", "question": "What were the GDP growth amounts for the first and second quarters respectively?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:376", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:376", "question": "What were the GDP growth amounts for the third and fourth quarters respectively?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['1981', '1982', '2001', '2002']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:377", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:377", "question": "What are the years specified in the pie chart of the document? Format all of the years in a list like ['2006', '2007', '2011']", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:378", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:378", "question": "How many figures are there in total in the article?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "14", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:379", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:379", "question": "As the Chairman of the Board who serves as the President of the Society of Indian Law Firms (Western Region), how many membership of Board of other companies does he have?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "83672770", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 18}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:afe620b9beac86c1027b96d31d396407.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:380", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:380", "question": "For the year ended March 31,2003, how much less were the shareholders' funds compared to the income in rupees?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:381", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:381", "question": "How many quizzes are there in the entire course?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:382", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:382", "question": "How many quizzes are there in units 4, 5, and 6 combined?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Florida Department of Health", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:383", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:383", "question": "Who produced the document that was revised on May 2016?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:384", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:384", "question": "Who produced the document that was revised on May 2018?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['strategic priority areas', 'strategies', 'objectives']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:385", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:385", "question": "What are the words written in the first rectangle on the top of the page two?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "By December 31,2018, reduce the average time to issue a facility license from 60 days (2015) to 45 days", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:386", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:386", "question": "WHAT IS THE 2nd OBJECTIVE OF REGULATORY EFFICIENCY?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Rick Scott", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:387", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:387", "question": "What is the name of the governor as mentioned on the first page of the document?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:388", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:388", "question": "What is the name of the governor as mentioned on the last page of the document?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "12", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:389", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:389", "question": "How many people are there in the images on the cover?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:390", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:390", "question": "How many cats are there in the images on page 1?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Leadership, Workforce and Infrastructure", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:391", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:391", "question": "What is the title of the diagram on page 9?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:392", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:392", "question": "What is the title of the diagram on page 10?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['LONG,HEALTHY LIFE', 'READINESS FOR EMERGING HEALTH THREATS']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:393", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:393", "question": "What are the top2 texts of the red words in the document?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:394", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:394", "question": "What are the top2 texts of the yellow words in the document?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:395", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:395", "question": "What is the address of Florida Department of Science?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['23', '21']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:396", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:396", "question": "How many strengths and weaknesses are metioned in Appendix C? Represent these two numbers as format of list.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:397", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:397", "question": "How many strengths and weaknesses are metioned in Appendix E?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:398", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:398", "question": "How many appendices appear in the document?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:e79deb02a0c0e87511080836c5d4347b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:399", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:399", "question": "How many people are there in total in the MQA Executive Leadership and the Prosecution Services Staff?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:400", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:400", "question": "How many people are there in total in the HAQ Executive Leadership?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:401", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:401", "question": "How many steps are involved in the binary fission of prokaryotic cells?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Duplication of chromosome and separation of copies', 'Continued elongation of the cell and movement of copies', 'Division into two daughter cell']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:402", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:402", "question": "What are the three steps of binary fission of a prokaryotic cell?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Chapter 8', 'Chapter 9', 'Chapter 10']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}, {"region": {"page": 48}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:403", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:403", "question": "Which chapters are included in the slide? Your answer should be formatted as a list like ['Chapter 1', 'Chapter 3'].", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "leopard", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:404", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:404", "question": "What animal is on the cover of each chapter?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:405", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:405", "question": "What plants is on the cover of each chapter?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:406", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:406", "question": "In the case presented in Chapter 9, what color are the third generation flowers produced by the purple and white flowers?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}, {"region": {"page": 46}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:407", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:407", "question": "How many full faces of people appear in the slide?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:408", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:408", "question": "How many cats appear in the slide?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Interphase', 'Prophase', 'Prometaphase', 'Metaphase', 'Anaphase', 'Telophase and cytokinesis']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:409", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:409", "question": "Which stages of cell division are shown on slides 12 and 14?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "ff", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:chapter8-geneticscompatibilitymode-141214140247-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:410", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:410", "question": "What is the genotype corresponding to attached earlobes?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:411", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:411", "question": "What is the genotype corresponding to black hair?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "milk giving black young buffalo 20 liter per day", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:412", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:412", "question": "What is the ebay.in listing for shown in the image?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "45", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:413", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:413", "question": "In the year in which the No of internet users was 330 million, what percentage of Indians were Debit Card users?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "91.77", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:414", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:414", "question": "How many more debit card users in millions were there in India in 2015 compared to 2014?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:415", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:415", "question": "How many more debit card users in millions were there in India in 2013 compared to 2012?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "3.91", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:416", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:416", "question": "What multiple of the 2014 e-commerce sales was achieved in 2018?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:417", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:417", "question": "What multiple of the 2014 e-commerce sales was achieved in 2016?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "399 million", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:418", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:418", "question": "How many people in India were using a debit card when the e-commerce sales reached 11 billion?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:419", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:419", "question": "How many people in India were using a debit card when the e-commerce sales reached 50 billion?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "14.7%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:420", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:420", "question": "What is the CAGR of television from 2012 to 2016?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:421", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:421", "question": "What is the CAGR of cars from 2012 to 2016?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "20", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 21}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}, {"region": {"page": 22}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:422", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:422", "question": "How many types of ecosystem players are listed in the slide?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "12", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:ecommerceopportunityindia-141124010546-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:423", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:423", "question": "How many 'Pioneers' and people on the cover of the slides are there in total in the slide?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:424", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:424", "question": "What is in the overlap area between Danger Zone and Machine Learning?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Hacking Skills', 'Substantive Expertise']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:425", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:425", "question": "The overlap of which two capabilities is considered the danger zone?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Dog', 'Cat', 'Bear']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:426", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:426", "question": "What types of animals appear in the PPT?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:427", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:427", "question": "What types of insects appear in the PPT?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Bengio', 'Hinton', 'LeCun']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 23}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:428", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:428", "question": "Which three deep learning conspirators appear in the PPT?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:429", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:429", "question": "How many times does the image of a bear appear in the PPT?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}, {"region": {"page": 30}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}, {"region": {"page": 32}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:431", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:431", "question": "How many layers does the neural network mentioned in the PPT for distinguishing faces have?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:432", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:432", "question": "How many layers does the neural network mentioned in the PPT for distinguishing cars have?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "Dog", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 58}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}, {"region": {"page": 60}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}, {"region": {"page": 61}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:433", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:433", "question": "Which animal did the detector ultimately recognize the person as?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Cat vs Dog dataset", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 34}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:434", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:434", "question": "Which dataset was introduced in the 'coding time' section of the PPT for training?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "9", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 42}, "source_id": "mmlongbench:catvsdogdlpycon15se-150512122612-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:435", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:435", "question": "How many libraries were imported in the code section of the PPT?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "71980", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:SnapNTell.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:436", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:436", "question": "Compared to ViQuAE, how many more QA pairs does SnapNtell have?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:437", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:437", "question": "Compared to MMDU, how many more QA pairs does SnapNtell have?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Eiffel Tower", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:SnapNTell.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:438", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:438", "question": "In the SnapNTell model architecture, what is the image used as an example input in the paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2885", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:SnapNTell.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:439", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:439", "question": "In the 3rd Wiki filtering, how many more entities were filtered out compared to the 1st Wiki filtering?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:440", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:440", "question": "In the 4rd Wiki filtering, how many more entities were filtered out compared to the 1st Wiki filtering?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "20", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:SnapNTell.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:SnapNTell.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:SnapNTell.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:SnapNTell.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:441", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:441", "question": "How many times do various sizes of the Eiffel Tower appear in the image?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:SnapNTell.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:SnapNTell.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:442", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:442", "question": "How many times does the image of The Acropolis Museum appear in the paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "676", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:SnapNTell.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:SnapNTell.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:443", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:443", "question": "How many years after The Chapel Bridge was The Acropolis Museum completed? Return me an integer.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Butterfly", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:RAR.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:444", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:444", "question": "In the pipeline diagram of the RAR model, which type of organism is used as the input case?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:445", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:445", "question": "In the pipeline diagram of the BERT model, which type of organism is used as the input case?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Mercedes-Benz E-Class Sedan", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:RAR.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:446", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:446", "question": "In the Ranking Prompt Example, what is the correct type of the car provided?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:447", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:447", "question": "In the Ranking Prompt Example, what is the correct type of the aircraft provided?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:RAR.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:448", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:448", "question": "In the example image for extending RAR to zero-shot recognition on object detection datasets, how many bounding boxes (bboxes) are there in total?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 21}, "source_id": "mmlongbench:RAR.pdf"}, {"region": {"page": 22}, "source_id": "mmlongbench:RAR.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:449", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:449", "question": "How many GPT-4V examples are listed in the paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:450", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:450", "question": "How many GPT-4o examples are listed in the paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "18.9%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 24}, "source_id": "mmlongbench:RAR.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:RAR.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:451", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:451", "question": "In the 4-shot classification task, how much higher is the top-5 average accuracy of RAR (LLaVA1.5) compared to the top-1 accuracy?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:RAR.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:RAR.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:452", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:452", "question": "In the visualization of the ranking examples and ranking prompt examples, how many images appear in total?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:STEPBACK.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:453", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:453", "question": "How many green bars appear in Figure 1?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "73.2%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:STEPBACK.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:454", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:454", "question": "What score did PaLM-2L + Step-Back achieve in MMLU Physics?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:455", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:455", "question": "What score did LLaVa + Step-Back achieve in MMLU Physics?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "15.9%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:STEPBACK.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:456", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:456", "question": "How much higher is the accuracy of PaLM-2L + RAG compared to PaLM-2L on TimeQA?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "16.7%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:STEPBACK.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:STEPBACK.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:457", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:457", "question": "How much lower is the accuracy of GPT-4 on SituatedQA compared to MMLU Chemistry?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:458", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:458", "question": "How much lower is the accuracy of GPT-4o on SituatedQA compared to MMLU Chemistry?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:STEPBACK.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:STEPBACK.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:459", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:459", "question": "How many pie charts are there in the paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "60.4%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:STEPBACK.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:STEPBACK.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:460", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:460", "question": "In the \"Error Analysis of Step-Back Prompting\", what is the sum of the baseline wrong percentages in TimeQA and MMLU high-school Physics?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:t480_ug_en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:461", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:461", "question": "How many connectors are there on the right side view of the computer?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "9", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 37}, "source_id": "mmlongbench:t480_ug_en.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:t480_ug_en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:462", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:462", "question": "How many trackpad gestures does this computer have?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "on the right", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 51}, "source_id": "mmlongbench:t480_ug_en.pdf"}, {"region": {"page": 52}, "source_id": "mmlongbench:t480_ug_en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:463", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:463", "question": "Which side of the camera indicator is on the infrared camera lens?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 119}, "source_id": "mmlongbench:t480_ug_en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:464", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:464", "question": "How many switches do you need to flip to remove the battery?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "SC", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 80}, "source_id": "mmlongbench:t480_ug_en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:465", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:465", "question": "Which country or region codes are needed to install Windows 10 in Mainland China?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:466", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:466", "question": "Which license codes are needed to install Windows 10 in Mainland China?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "Pb", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 160}, "source_id": "mmlongbench:t480_ug_en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:467", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:467", "question": "Which chemical element in the hard disk exceeds the limit requirements stipulated in the GB/T 26572 standard?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:468", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:468", "question": "Which chemical element in the earphone exceeds the limit requirements stipulated in the GB/T 26572 standard?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 150}, "source_id": "mmlongbench:t480_ug_en.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:469", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:469", "question": "How many UltraConnect wireless antennas does this computer have?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "very cold milk (up to 8 degrees celsius)", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:470", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:470", "question": "What temperature does the green color of the coffee machine represent for the milk?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:471", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:471", "question": "What temperature does the yellow color of the coffee machine represent for the milk?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "16", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 24}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:472", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:472", "question": "How many grams of coffee can be put in the manual insert? Return me an integer.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "11", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:473", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:473", "question": "How many types of main menu functions are there?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "15:00", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 50}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:474", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:474", "question": "In the picture on page 50, what time does the blue bar start?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:475", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:475", "question": "In the picture on page 50, what time does the red bar start?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "9", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 56}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}, {"region": {"page": 57}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:476", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:476", "question": "How many steps are there for data exchange via USB?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:477", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:477", "question": "How many coffee quality levels can the coffee machine be set to?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Boiler: over-temperature ', 'Heater error ']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 88}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}, {"region": {"page": 89}, "source_id": "mmlongbench:User_Manual_1500S_Classic_EN.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:478", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:478", "question": "What are the error descriptions corresponding to error numbers 88 and 188?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:479", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:479", "question": "What are the error descriptions corresponding to error numbers 288 and 388?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "red", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:camry_ebrochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:480", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:480", "question": "What color is the car on the cover?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:481", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:481", "question": "What color is the train on the cover?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:camry_ebrochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:482", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:482", "question": "How many cars appear on page three?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "4G", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:camry_ebrochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:483", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:483", "question": "What technology does the car’s Wi-Fi Connect use?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:camry_ebrochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:484", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:484", "question": "How many types of wheels are introduced in total?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['SE', 'SE Nightshade', 'XSE', 'XSE6', 'TRD', 'LE Hybrid']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:camry_ebrochure.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:camry_ebrochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:485", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:485", "question": "What models of cars are introduced on pages 10 and 11?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:486", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:486", "question": "What models of wheel are introduced on pages 10 and 11?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:camry_ebrochure.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:camry_ebrochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:487", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:487", "question": "How many people appear on pages three and four?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "37%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:q1-2023-bilibili-inc-investor-presentation.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:488", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:488", "question": "By what percentage has gross profit increased in Q1 of 2023 compared to Q1 of 2022?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:489", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:489", "question": "By what percentage has gross profit increased in Q1 of 2021 compared to Q1 of 2020?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "86%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:q1-2023-bilibili-inc-investor-presentation.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:490", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:490", "question": "What percentage of users are under 35?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:491", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:491", "question": "What percentage of users are under 25?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:q1-2023-bilibili-inc-investor-presentation.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:492", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:492", "question": "How many triangles appear on the eighth side?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['dog', 'cat']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:q1-2023-bilibili-inc-investor-presentation.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:493", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:493", "question": "What animals appear on page nine? Enumerate them in a list.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "5%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:q1-2023-bilibili-inc-investor-presentation.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:q1-2023-bilibili-inc-investor-presentation.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:494", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:494", "question": "Compared with 22Q1, in 23Q1, how much higher is the increase rate of number of daily average active content creators than average daily video views?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:q1-2023-bilibili-inc-investor-presentation.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:q1-2023-bilibili-inc-investor-presentation.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:495", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:495", "question": "How many times does mobile phone appear on pages 16 and 18?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:Pew-Research-Center_Hispanic-Identity-Report_12.20.2017.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:496", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:496", "question": "What's the value of the smallest bar in the graph about the percentage of U.S. adults with Hispanic ancestry who self-identify as Hispanic or Non-Hispanic?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "21", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:Pew-Research-Center_Hispanic-Identity-Report_12.20.2017.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:497", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:497", "question": "What's the average value of all orange bars in the chart about the percentage of U.S. adults with Hispanic ancestry who self-identify as Hispanic or Non-Hispanic?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Foreign born (excl. PR)", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 26}, "source_id": "mmlongbench:Pew-Research-Center_Hispanic-Identity-Report_12.20.2017.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:498", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:498", "question": "According to the 2015 National Survey of Latinos details, which group of Latinos has the largest sample size among the 1500 samples in total?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "65%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:Pew-Research-Center_Hispanic-Identity-Report_12.20.2017.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:Pew-Research-Center_Hispanic-Identity-Report_12.20.2017.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:499", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:499", "question": "According to the Pew Research Center 2015 National Survey of Latinos, what is the percentages of Foreign born Latinos most often uses the name of their origin country to describe themselves?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Gray", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:Pew-Research-Center_Hispanic-Identity-Report_12.20.2017.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:500", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:500", "question": "What's the color of the Third of higher generation bar in the chart about Immigrant generations and U.S. Latinos?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "56.34%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 21}, "source_id": "mmlongbench:Pew-Research-Center_Hispanic-Identity-Report_12.20.2017.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:501", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:501", "question": "What's the averaged percentage of Latinos (Foreign born, Second generation, Third or higher generation) say all or most of their friends are Latinos?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Cubans", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 73}, "source_id": "mmlongbench:PRE_2022.09.29_NSL-politics_REPORT.pdf"}, {"region": {"page": 95}, "source_id": "mmlongbench:PRE_2022.09.29_NSL-politics_REPORT.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:502", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:502", "question": "Which Hispanic origin group in the United States is more likely to have a positive impression of capitalism and be dissatisfied with the nation's direction?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Puerto Rican", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 57}, "source_id": "mmlongbench:PRE_2022.09.29_NSL-politics_REPORT.pdf"}, {"region": {"page": 66}, "source_id": "mmlongbench:PRE_2022.09.29_NSL-politics_REPORT.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:503", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:503", "question": "Which Hispanic origin group in the United States is most likely to vote for Republican candidate for the U.S. House of Representatives in their district and to say Trump should run for president in 2024?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "46%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 97}, "source_id": "mmlongbench:PRE_2022.09.29_NSL-politics_REPORT.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:504", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:504", "question": "What proportion of the Spanish dominant Latinos express a negative impression of socialism?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Cubans', 'Central Americans']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 86}, "source_id": "mmlongbench:PRE_2022.09.29_NSL-politics_REPORT.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:506", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:506", "question": "Which Hispanic origin groups with less than 60% holding the view that People NOT seeing racial discrimination where it really DOES exist? Return me a list please.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Little Rock", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:507", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:507", "question": "What's the geographic market name for EPS Code AR01?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:508", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:508", "question": "According to this document, what's the geographic market name for EPS Code AR03?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:509", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:509", "question": "How many geographic markers in Greater Houston, Texas?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:510", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:510", "question": "According to this document, how many geographic markers in the Oxford?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Start new search", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 18}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:511", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:511", "question": "What is the Word written in Orange box on page 17?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:512", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:512", "question": "What is the word written in Green box on page 17?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "13", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:513", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:513", "question": "How many geographic market names are listed under Los Angeles area?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "12", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:514", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:514", "question": "How many geographic markets in Texas belongs to the Major Metropolitan Area?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}, {"region": {"page": 2}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:515", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:515", "question": "How many recorded videos in the Recorded Demonstration section?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "No", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:516", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:516", "question": "In the context of the College Board Search Platform, is market CA03 considered part of the San Francisco Bay Area? Answer 'yes' or 'no'.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Covina and West Covina", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:517", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:517", "question": "Within the geographic market segments defined by the College Board Search platform, which of the two designated areas, CA19 or Covina and West Covina, encompasses a larger geographic region?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Right", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:8dfc21ec151fb9d3578fc32d5c4e5df9.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:518", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:518", "question": "On the left or right side of the screen, can you immediately find information about the percentage of the total available students represented by your chosen search criteria? Answer 'left' or 'right'.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "eMarketer", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:519", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:519", "question": "What is the Source of the data for the audience of the platform whose metrics are page likes, post clicks, video views, post reach, hide, report as spam, unlikes, 30-second views++, post types, engagement: reactions, shares, comments, and FB referral traffic?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:520", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:520", "question": "According to this slides, What is the Source of the data for the audience of Google?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:521", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:521", "question": "According to this slides, what age range accounts for the most users of the Amazon?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "4257", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:522", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:522", "question": "How many likes does the post by Disneyland have on the platform with an audience of 10.6 for Age 65+?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:523", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:523", "question": "How many dislikes does the post by Disneyland have on the platform with an audience of 10.6 for Age 65+?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "2013", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:524", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:524", "question": "In the two years 2013-2014, in which year is there a lower percentage of users from rural locations on the platform whose measurement tools include Twitter Analytics, Followerwonk, Buffer, and TweetReach?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "32", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:525", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:525", "question": "In 2013, what is the total percentage of online adult users in the \"White, Non-Hispanic\" and \"Hispanic\" categories on the platform whose metrics include Followers, Tweet Clicks, Video Views, Tweet Reach, Engagement Rate, Completion Rate, Engagement: Mentions, Retweets, Likes, Top Tweet, Top Mention, Top Follower, Top Media Tweet, and Twitter Referral Traffic?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Target", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:526", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:526", "question": "Which account tweets content tagged #WednesdayWisdom?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "62%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}, {"region": {"page": 24}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:527", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:527", "question": "What is the total percentage of `college graduate` and `high school or less` on the platform whose measurement tools include Google Analytics?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:528", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:528", "question": "What is the total percentage of `elementary school` and `phd` on the platform whose measurement tools include Google Analytics?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Post Reach', 'Video Views']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}, {"region": {"page": 23}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:529", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:529", "question": "Which metrics are NOT included in LinkedIn Metrics: Post Reach, Post Types, Video Views?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "44%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:measuringsuccessonfacebooktwitterlinkedin-160317142140_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:530", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:530", "question": "What is the total percentage of income > 75k for the LinkedIn platform?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "00:04:14", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:amb-siteaudits-ds15-150204174043-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:531", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:531", "question": "What is the average session duration for desktop?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:532", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:532", "question": "According to the slides, what is the average session duration for TV?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Direct", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:amb-siteaudits-ds15-150204174043-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:533", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:533", "question": "According to the multi-channel conversion visualizer, which channel led to the most conversions?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "AWS", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:amb-siteaudits-ds15-150204174043-conversion-gate01_95.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:amb-siteaudits-ds15-150204174043-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:534", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:534", "question": "Which of the following company logos do not appear in the slides? Facebook, AWS, Cargo and Manta.", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 23}, "source_id": "mmlongbench:amb-siteaudits-ds15-150204174043-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:535", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:535", "question": "How many questions are there about URL timeout issues?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Recreation / Travel", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 28}, "source_id": "mmlongbench:amb-siteaudits-ds15-150204174043-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:536", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:536", "question": "Which category has the most topical trust flows?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "137", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 25}, "source_id": "mmlongbench:amb-siteaudits-ds15-150204174043-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:537", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:537", "question": "How many pages for the hatom data type in the Structured Markup?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "/video/videocat/video9201", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:amb-siteaudits-ds15-150204174043-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:538", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:538", "question": "What is the Top-Level Page name of the page with the slowest page processing time?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Blue", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 153}, "source_id": "mmlongbench:2309.17421v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:539", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:539", "question": "In Figure 122, what's the colour of OSCARb in the prompt?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "YouTube Music", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 143}, "source_id": "mmlongbench:2309.17421v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:540", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:540", "question": "In Figure 113, what's the name of the music app shown in the computer screen?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "$49.99", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 141}, "source_id": "mmlongbench:2309.17421v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:541", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:541", "question": "In Figure 111, what's the price of the keyboard shown in the screenshot?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "76%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 137}, "source_id": "mmlongbench:2309.17421v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:542", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:542", "question": "In Figure 107, what's the battery percentage shown in the screenshot?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Ukraine", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 131}, "source_id": "mmlongbench:2309.17421v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:543", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:543", "question": "In Figure 101, which country is the person on the cover of the news on the politico website from?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "JoooDeee", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 118}, "source_id": "mmlongbench:2309.17421v2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:544", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:544", "question": "In Figure 89, what is the brand name of the coffee machine in the picture?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "92", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 26}, "source_id": "mmlongbench:2005.12872v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:545", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:545", "question": "According to the DETR PyTorch inference code, what is the output dimension of the linear_class layer?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Person", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 25}, "source_id": "mmlongbench:2005.12872v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:546", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:546", "question": "When number of visible instances = 60, which category has the largest proportion of missed instances?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "57.3", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:2005.12872v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:547", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:547", "question": "What's the AP50 value of DETR with L1 loss and without GIoU loss on the COCO validation set?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:548", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:548", "question": "What's the AP50 value of DETR without L1 loss and without GIoU loss on the COCO validation set?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "Red", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:2005.12872v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:549", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:549", "question": "According to Fig 10 architecture of DETR’s transformer, what is the colour of the Multi-Head Self-Attention layer?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "36", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:2005.12872v3.pdf"}, {"region": {"page": 22}, "source_id": "mmlongbench:2005.12872v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:550", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:550", "question": "How many multi-head self-attention layers does DETR's default structure have?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "+65 6778-0808", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 43}, "source_id": "mmlongbench:StudentSupport_Guidebook.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:551", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:551", "question": "According to the PDF source, what is the telephone number of Prime Taxi?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:552", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:552", "question": "According to the PDF source, what is the telephone number of Uber Taxi?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "179", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:StudentSupport_Guidebook.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:553", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:553", "question": "What is the bus route number that appears in the figure of this document?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 28}, "source_id": "mmlongbench:StudentSupport_Guidebook.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:554", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:554", "question": "According to the document, how many SG University Holidays from 15 May 2016 to 15 Jan 2017?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "STARBUCKS", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 34}, "source_id": "mmlongbench:StudentSupport_Guidebook.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:555", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:555", "question": "What is the coffee brand name shown in figure of the document?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['OCBC', 'citibank', 'DBS']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 34}, "source_id": "mmlongbench:StudentSupport_Guidebook.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:556", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:556", "question": "What is the bank names shown in the figure of this document?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Blue", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 117}, "source_id": "mmlongbench:mmdetection-readthedocs-io-en-v2.18.0.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:557", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:557", "question": "What's the color of the 'loss_cls' line in the loss curve image?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "DecoupledSOLOHead", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 336}, "source_id": "mmlongbench:mmdetection-readthedocs-io-en-v2.18.0.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:558", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:558", "question": "If I want to use the detector in the paper `SOLO: Segmenting Objects by Locations`, what is the implemented class name in `mmdet.models.dense_heads`?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:559", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:559", "question": "If I want to use the detector in the paper `V3Det: Vast Vocabulary Visual Detection Dataset`, what is the implemented class name in `mmdet.models.dense_heads`?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 75}, "source_id": "mmlongbench:mmdetection-readthedocs-io-en-v2.18.0.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:560", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:560", "question": "How many data preparation operators in the classical pipeline?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "YOLACT", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 229}, "source_id": "mmlongbench:mmdetection-readthedocs-io-en-v2.18.0.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:561", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:561", "question": "Which paper uses the Fast NMS processing?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['46', '27', '64']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:12-15-15-ISIS-and-terrorism-release-final.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:12-15-15-ISIS-and-terrorism-release-final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:562", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:562", "question": "In 2015, how many percentage of surveyed adults, Republicans, and Democrats believed that the goverment was doing very/fairly well in reducing the threat of terrorism? Please write the answer in list format, e.g., [\"3\",\"2\"]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:563", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:563", "question": "In 2020, how many percentage of surveyed adults, Republicans, and Democrats believed that the goverment was doing very/fairly well in reducing the threat of terrorism? Please write the answer in list format, e.g., [\"3\",\"2\"]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "Iran's nuclear program", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:12-15-15-ISIS-and-terrorism-release-final.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:12-15-15-ISIS-and-terrorism-release-final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:564", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:564", "question": "In 2015, among all those that are considered as bigger threats to well-being of the United States compared to August 2014, which one has the biggest R-D difference?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:565", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:565", "question": "In 2015, among all those that are considered as bigger threats to well-being of the China compared to August 2014, which one has the biggest R-D difference?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:12-15-15-ISIS-and-terrorism-release-final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:566", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:566", "question": "How many bar charts are in the report?Please write the answer in float format.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "will definitely/probably succeed", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 18}, "source_id": "mmlongbench:12-15-15-ISIS-and-terrorism-release-final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:567", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:567", "question": "although few says that current military effort in Iraq and Syria is going well,  what are the more people's attitude in still believe that the U.S. and its allies in the ISIS flight in the end in 2015?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1954", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:e639029d16094ea71d964e2fb953952b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:576", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:576", "question": "Since what year has Mr. Kildee been involved with child nutrition?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:577", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:577", "question": "What is the yellow color italic texts in page 9?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "2003-10-07", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:e639029d16094ea71d964e2fb953952b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:578", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:578", "question": "What date is mentioned at the beginning of page(1)? Format the date as YYYY-MM-DD", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "no", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:e639029d16094ea71d964e2fb953952b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:579", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:579", "question": "Is there a signature present on the last page? Directly answer 'yes' or 'no'.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:e639029d16094ea71d964e2fb953952b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:580", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:580", "question": "How many websites are present on the cover page?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:e639029d16094ea71d964e2fb953952b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:581", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:581", "question": "What page has an eight-pointed snowflake image? Give me an integer.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:e639029d16094ea71d964e2fb953952b.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:e639029d16094ea71d964e2fb953952b.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:582", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:582", "question": "How many committee members from Texas are presented in the conference of improving the quality and efficiency of commodity distribution to federal child nutrition programs?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2021-02-08", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:583", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:583", "question": "Format the date mentioned on page 14 as YYYY-MM-DD.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "514-312-0292", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:585", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:585", "question": "What is INF SERCRL LLP FAX No on page fourteen?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:586", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:586", "question": "What is INF SERCRL LLP FAX No on page thirteen?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "SUPERIOR COURT", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:587", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:587", "question": "What type of court is noted on the cover page?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "lnahmiash@infavocats.com", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:588", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:588", "question": "what is the email id of the mtre laurent nahmiash ?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:589", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:589", "question": "what is the texts of the underlined italic words in page 2?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "27", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 2}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:590", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:590", "question": "How many Exhibits are mentioned in the document?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['1', '2', '3', '10']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 2}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:a5879805d70c854ea4361e43a84e3bb2.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:592", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:592", "question": "How many times does Exhibit P-10 mentioned in the document? List all pages and formatted as a list, for example, [\"1\",\"2\"]", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Special Agent Marc Silski", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:3276a5b991c49cf5f9a4af0f7d6fce67.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:593", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:593", "question": "Which officer verify and signed the complaint?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['$35,000', '$40,684']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:3276a5b991c49cf5f9a4af0f7d6fce67.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:594", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:594", "question": "What are the amounts on checks issued to the Mont Blanc company? Enumerate each amount within a list.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "239516", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:3276a5b991c49cf5f9a4af0f7d6fce67.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:595", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:595", "question": "what is the CA, Bar no of the report?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:596", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:596", "question": "According to the complaint, who was interviewed by Traveler's Insurance Companay agents on Oct. 24, 2016?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['13', '14']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:3276a5b991c49cf5f9a4af0f7d6fce67.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:3276a5b991c49cf5f9a4af0f7d6fce67.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:597", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:597", "question": "Which pages have information blacked out? List the page numbers in a list like [2, 3, 5].", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "19", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:3276a5b991c49cf5f9a4af0f7d6fce67.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:3276a5b991c49cf5f9a4af0f7d6fce67.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:598", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:598", "question": "What is the sum of numbers that are written manually on the page 15 and page 16? Write the answer in the float format.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:3276a5b991c49cf5f9a4af0f7d6fce67.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:3276a5b991c49cf5f9a4af0f7d6fce67.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:600", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:600", "question": "How many signatures are appeared on the page 15 and page 16? Write the answer in the float format.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "100%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:601", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:601", "question": "What percentage of respondents of the sector in which 15% are doing promotions to customers over Wi-Fi use wifi at stores?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:602", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:602", "question": "What percentage of respondents of the sector in which 15% are doing promotions to customers over cellphone use wifi at stores?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "Vice President of Product Alliances", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 42}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 43}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:603", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:603", "question": "What is the job of the contact person in the picture at the top of page 42?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:604", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:604", "question": "What is the job of the contact person in the picture at the top left of page 42?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "+1-615-591-2955", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 42}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 43}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:605", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:605", "question": "Besides the two companies on page 42, what is the contact phone number of another company? Please write in string format.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:606", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:606", "question": "Besides the two companies on page 42, what is the contact phone number of another two companies? Please write in string format.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "122", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 37}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:607", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:607", "question": "What is the sum of percentage of customers and employees said that Wi-Fi increases customer loyalty in hospitality in 2015?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:608", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:608", "question": "What is the sum of percentage of customers and employers said that Wi-Fi increases customer loyalty in hospitality in 2015?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['5', '24', '25', '26', '27', '28', '31', '44']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 24}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 31}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 44}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:609", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:609", "question": "How many women in total are there in the slides? Please list the page number in the list format, e.g., [\"1\",\"2\"]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:610", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:610", "question": "How many girls in total are there in the slides? Please list the page number in the list format, e.g., [\"1\",\"2\"]", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "85%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:earthlinkweb-150213112111-conversion-gate02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:611", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:611", "question": "What percentage of the second largest demographic segment of respondents reported not conducting promotions to customers over Wi-Fi in 2015?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:612", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:612", "question": "What percentage of the second largest demographic segment of respondents reported not conducting promotions to customers over Wi-Fi in 2020?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "Lower thoracic esophagus", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:613", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:613", "question": "What part of the esophagus is just above the cardioesophageal junction as seen in the figure?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:614", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:614", "question": "How many words for parts that start with 'A' are in the figure on slide 11?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:615", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:615", "question": "How many words for parts that start with 'X' are in the figure on slide 11?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "bottom", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 77}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:616", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:616", "question": "What position is the man with the red shirt in the figure on the last page, directly answer 'bottom', 'middle' or 'top'?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:617", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:617", "question": "What position is the woman with the red shirt in the figure on the last page—bottom, middle, or top?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 27}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}, {"region": {"page": 29}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:618", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:618", "question": "How many numbers are mentioned on Slide 27 and the slide two positions after it?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 37}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}, {"region": {"page": 56}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}, {"region": {"page": 58}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}, {"region": {"page": 59}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}, {"region": {"page": 64}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}, {"region": {"page": 65}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}, {"region": {"page": 73}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:619", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:619", "question": "How many slides contain the type of pictures that are two X-ray film distributed side by side?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 48}, "source_id": "mmlongbench:bariumswallowpresentation-090810084400-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:620", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:620", "question": "How many slides contain the type of pictures that are two X-ray film distributed vertically?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2007", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:competitiveoutcomes-091006065143-phpapp01_95.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:competitiveoutcomes-091006065143-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:621", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:621", "question": "What year is printed on the t-shirt the man is wearing in the photograph in which he is holding a serving plate full of what gives a utility of +10 from consuming 1 assuming standard consumption economics?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 42}, "source_id": "mmlongbench:competitiveoutcomes-091006065143-phpapp01_95.pdf"}, {"region": {"page": 43}, "source_id": "mmlongbench:competitiveoutcomes-091006065143-phpapp01_95.pdf"}, {"region": {"page": 48}, "source_id": "mmlongbench:competitiveoutcomes-091006065143-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:622", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:622", "question": "How many website URLs are contained in the slides?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "+10", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:competitiveoutcomes-091006065143-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:623", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:623", "question": "What is the utility derived from each hot dog that the boy is eating in the picture in slide 4?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:624", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:624", "question": "What is the utility derived from each hot dog that the boy is eating in the picture in slide 6?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:competitiveoutcomes-091006065143-phpapp01_95.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:competitiveoutcomes-091006065143-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:625", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:625", "question": "How many pictures of musical instruments are contained in the slides?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "204", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:626", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:626", "question": "For the dataset that uses Prover9 as the symbolic solver, how many examples in this dataset are used for evaluation?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:627", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:627", "question": "For the dataset that uses HOVER as the symbolic solver, how many examples in this dataset are used for evaluation?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:628", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:628", "question": "For all the examples given in Figure 5 and Figure 6, how many of them Logic-LM predicted the correct answer?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:629", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:629", "question": "For all the examples given in Figure 7 and Figure 8, how many of them Logic-LM predicted the correct answer?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "15", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:630", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:630", "question": "How many prompt examples are given in the Appendix?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:631", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:631", "question": "When using GPT-4 as the base language model, how many datasets does Logic-LM (without self-refinement) outperform the two baseline models in?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Pyke", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:632", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:632", "question": "In this paper, what symbolic solver is used to implement the logic programming module shown in Figure 1?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "ProofWriter", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:2023.findings-emnlp.248.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:633", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:633", "question": "Which dataset used in this paper was proposed in 2022 and all of its logical reasoning problems are multiple-choice questions with 5 options?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "post-hoc correction", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:tacl_a_00660.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:643", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:643", "question": "Among the three correction strategies: training-time correction, generation-time correction, and post-hoc correction, which one has the most representative papers in the survey?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:644", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:644", "question": "Among the three generation strategies: training-time generation, generation-time generation, and post-hoc generation, which one has the most representative papers in the survey?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['3', '2', '3']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:tacl_a_00660.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:645", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:645", "question": "How many strategies are plotted for training-time correction, generation-time correction, and post-hoc correction, respectively? Write the answer in the list format, e.g., [\"1\",\"2\"]", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:646", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:646", "question": "How many strategies are plotted for training-time generation, generation-time generation, and post-hoc generation, respectively? Write the answer in the list format, e.g., [\"1\",\"2\"]", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:tacl_a_00660.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:647", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:647", "question": "For the papers that adopted the Self-Refine strategy in Table 2, how many of them are not mentioned in the \"Self-Correction\" section of page 9? Write the answer in int format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:tacl_a_00660.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:648", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:648", "question": "For the papers that adopted the Model Debate strategy in Table 2, how many of them are not mentioned in Section 3.4? Write the answer in float format.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:tacl_a_00660.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:649", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:649", "question": "For all the papers that are listed in Table 1 and Table 2, how many of them are proposed in 2021?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Yes", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:tacl_a_00660.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:tacl_a_00660.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:650", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:650", "question": "Does Section 3.3 cite more papers than Section 3.1? Directly answer 'yes' or 'no'.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Fudan University', 'Shanghai Jiao Tong Univesity', 'University of Hong Kong.']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:651", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:651", "question": "List all the Chinese universities that have a student exchange programme with FASS. Write the answer in alphabetical order.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:652", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:652", "question": "List all the Chinese universities that have a student exchange programme with SoC. Write the answer in alphabetical order.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "Department of History", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:653", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:653", "question": "Which of the following department does not provide a PhD programme, Department of Economics, Department of English Language and Literature, or Department of History?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:654", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:654", "question": "Which of the following department does not provide a MBA programme, Department of Economics, Department of English Language and Literature, or Department of History?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "Cultural Studies in Asia, Department of Psychology", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:655", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:655", "question": "Which department or programme has the most number of people that appears in the guidebook to advertise for it?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:656", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:656", "question": "For all the people that appear in the guidebook to speak for their department or programmes, how many of them are assistant professors?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "17", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 21}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 22}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 23}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 24}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 29}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 30}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}, {"region": {"page": 31}, "source_id": "mmlongbench:NUS-FASS-Graduate-Guidebook-2021-small.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:657", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:657", "question": "How many departments/programmes are introduced in the \"Research & Course Work Programmes@FASS\" part (page 11-31)?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:DSA-278777.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:658", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:658", "question": "How many parts has the prefix N in the packages?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "P1.1", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:DSA-278777.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:659", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:659", "question": "Which port has the alternative function that capture Trigger from port 0-3?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:660", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:660", "question": "Which port has the alternative function that capture Trigger from port 10-13?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "0.0", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:DSA-278777.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:DSA-278777.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:661", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:661", "question": "What is the minimum voltage of the symbol at the bottom left corner in the Figure of EPROM programming and verification waveforms? Write down the answer in float format.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['12', '13', '16', '18', '20']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:DSA-278777.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:DSA-278777.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:DSA-278777.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:DSA-278777.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:DSA-278777.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:662", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:662", "question": "How many waveform figures are contained in the guidebook for 272318? List the page numbers in the list format in ascending order,e.g., [\"1\",\"2\"]", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:DSA-278777.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:DSA-278777.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:663", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:663", "question": "How many connection graphs are contained in the guidebook?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Yes", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:owners-manual-2170416.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:owners-manual-2170416.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:664", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:664", "question": "In the two styles shown in the \"Parts and Features\" section, are both styles have the egg bin in the picture? Directly answer 'yes' or 'no'.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:665", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:665", "question": "In the two styles shown in the \"Parts and Features\" section, are both styles have the orange in the picture?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "after", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:owners-manual-2170416.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:owners-manual-2170416.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:666", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:666", "question": "In the \"Before Using Your Refrigerator\" Section, \"Installing it properly\" is the step before or after the step \"Plugging it in\"?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Crisper", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:owners-manual-2170416.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:owners-manual-2170416.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:667", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:667", "question": "Which creation has more steps, To remove the drop-in trim piece or to remove the crisper?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:owners-manual-2170416.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:668", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:668", "question": "How many steps are required to replace the grille? Write the answer in float format.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Elastic Compute Service", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:669", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:669", "question": "What is the central component of the figure at page 17?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:670", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:670", "question": "What is the central component of the table at page 17?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 21}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:671", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:671", "question": "How many ECS components wiil the AliCloud DNS go through in the figure at Page 18?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:672", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:672", "question": "How many ECS components wiil the BaiduCloud DNS go through in the figure at Page 18?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "3.0", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:673", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:673", "question": "How many cameras outside the China area does the figure at Page 19 has? Write the answer in float format.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "30", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}, {"region": {"page": 26}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:674", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:674", "question": "How many website url does the document have?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "18", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:675", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:675", "question": "How many data centers does the China's largest cloud provider have?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Dubai', 'Frankfurt', 'Hong Kong', 'London', 'New York', 'Paris', 'San Mateo', 'Seoul', 'Singapore', 'Sydney', 'Tokyo']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:Campaign_038_Introducing_AC_Whitepaper_v5e.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:676", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:676", "question": "Which cities does the China's largest cloud provider have teams? Write the answer in list format with alphabetical rder.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['GS5002', 'GS6001', 'GS6883A', 'GS5101']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:ISEP_student_handbook_2020.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:677", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:677", "question": "Which compulsory ISEP courses does the students must have? Write the answer in list format in ascending order.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:678", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:678", "question": "Which compulsory SoC courses does the students must have? Write the answer in list format in ascending order.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:679", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:679", "question": "What is the first date for January intake students to start the second FYP project?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:ISEP_student_handbook_2020.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:ISEP_student_handbook_2020.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:ISEP_student_handbook_2020.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:680", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:680", "question": "How many areas do the course requiremnt have?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "10", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:ISEP_student_handbook_2020.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:ISEP_student_handbook_2020.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:681", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:681", "question": "What is the maximum hours of ISEP buddy scheme does a Singaporean ISEP students require to do?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Jun', 'Aug']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:ISEP_student_handbook_2020.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:682", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:682", "question": "What is the deadline  month of the January intake ISEP students need to nominate supervisors and nominations of tac? Write the answer in list format, e.g., [\"Jan\",\"Feb\"]", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "92", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:683", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:683", "question": "What is the highest point of % of voters who say there was more mudslinging than in past elections?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "men", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:684", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:684", "question": "Do more men or women (in %) think a female president will be elected in a lifetime?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:685", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:685", "question": "Do more White or Black Americans (in %) think a female president will be elected in a lifetime?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}, {"region": {"page": 33}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}, {"region": {"page": 34}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}, {"region": {"page": 35}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:686", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:686", "question": "How many line plots are there in the report?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Clinton", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 28}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}, {"region": {"page": 29}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:687", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:687", "question": "Who has a higher median campaign grade? Trump or Clinton?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}, {"region": {"page": 34}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:688", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:688", "question": "How many charts are related to mudslinging?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "[53, 1.4]", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:689", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:689", "question": "How many % of voters reactions are \"uneasy\" and \"excited\" about Trump's victory?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "31%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:11-21-16-Updated-Post-Election-Release.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:690", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:690", "question": "How many % of voters reactions are \"angry\" about Trump's victory?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:691", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:691", "question": "How many line plots are in this report?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "49", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:692", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:692", "question": "How many percent of people cite not enough timely testing as a major reason COVID-19 outbreak has continued?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "12", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:693", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:693", "question": "How many charts depict partisan differences?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "50", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:694", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:694", "question": "How many percent of White Americans disapprove of the way Trump does as president?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:695", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:695", "question": "How many percent of American Japanese disapprove of the way Trump does as president?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "[62, 30]", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:PP_2020.08.06_COVID-19-Restrictions_FINAL-1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:696", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:696", "question": "How many % of Rep/Lean Rep people think cases have risen primally because of more testing and how many % of Rep/Lean Rep people think the federal government should be primarily responsible for COVID-19 control policy?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:697", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:697", "question": "How many % of Age 18-29 people think cases have risen primally because of more testing and how many % of Age 18-29 people think the federal government should be primarily responsible for COVID-19 control policy?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:698", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:698", "question": "Are there counties in Florida with more than 100 deaths per 100,000 residents?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:699", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:699", "question": "What types of charts are in the document?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['48897809', '5152012']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 1}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:700", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:700", "question": "Write the filling id and case number in this document?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:701", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:701", "question": "What is the name of the person who signed at the end of the opinion?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "FN54. Wong v. Allison, 208 F.3d 224, 2000 WL 206572", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:702", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:702", "question": "Which case is related to the statement that \"PIC also never determined if Hanson's qualified immunity defense would overcome her conflicts of interest.\"?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Scott Bradley.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 2}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:703", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:703", "question": "The document represents which judges' opinions?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "DELAWARE STATE PUBLIC INTEGRITY COMMISSION.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:7c3f6204b3241f142f0f8eb8e1fefe7a.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:704", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:704", "question": "Who is the defendant of this case?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "13", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:713", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:713", "question": "How many pages contain tables?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2008", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:714", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:714", "question": "WHAT IS THE LAST YEAR PRESENT IN THE TABULUAR COLUMN?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Development and integration", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:715", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:715", "question": "What is the FEA SRM Service type for Instrumentation and testing?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Mercury Interactive functional testing', 'Mercury Interactive Installation Testing', 'Mercury Interactive load/stress/volume testing', 'Mercury Interactive security and access control testing', 'oracle microsoft sql server mssql', 'usability testing (508 testing)']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:716", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:716", "question": "Name the list of service specification that comes under test management?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Software Development", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:717", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:717", "question": "What agency component costs the most of fund in the SRM Table?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Microsoft Oracle Open Database Connectivity (ODBC)', 'Microsoft Online Analytical Processing (OLAP)', 'XML for Analysis', 'Microsoft Active Server Pages (ASP)', 'Microsoft Active Server Pages .Net (ASP.Net)', 'System to System', 'Internal (within Agency)']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:718", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:718", "question": "What service specifications are associated with the SRM Component of Information Sharing?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "N/A", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:936c0e2c2e6c8e0c07c51bfaf7fd0a83.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:719", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:719", "question": "Describe the significant changes of the Risk Management Plan since last year.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "25mm Sheeting", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:720", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:720", "question": "Is 20mm Sheeting or 25mm Sheeting an appropriate size for timber formwork?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "14", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:721", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:721", "question": "How many engineering diagrams are shown to explain the formwork for columns? Any instance is considered as a separate diagram.", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "[\"Stage 5\"]", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 41}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 42}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 43}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 44}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:722", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:722", "question": "Which stages of casting a tunnel framework require a heater?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:723", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:723", "question": "Which stages of casting a tunnel framework require a cooler?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['A large wall formwork system', 'A staircase formwork', 'A gang formwork system']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 50}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 51}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 52}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:724", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:724", "question": "What structures are shown as examples of prefabricated formwork?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 55}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:725", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:725", "question": "How many days of rapid hardening cement are needed for the first structural member shown in the slides?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Circular and octagonal columns', 'Column bracing formwork']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:formwork-150318073913-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:726", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:726", "question": "What are the special forms of column formworks that are illustrated with diagrams in the slides?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "12", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:727", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:727", "question": "How many more Tweets are attributed to Germanwings than Lufthansa?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:728", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:728", "question": "How many more Tweets are attributed to Germanwings than Boeing?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "41", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:729", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:729", "question": "How many Tweets are attributed to Germanwings, Lufthansa, and Airbus in total?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:730", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:730", "question": "How many Tweets are attributed to Boeing?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['yellow', 'black and white']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:731", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:731", "question": "What colors are the Lufthansa logo within the first 90 minutes and afterward until 120 minutes?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:732", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:732", "question": "How many figures are related to Airbus?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "14:04 CET", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:733", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:733", "question": "When did the number of tweets referencing Germanwings exceed 200,000? Your report time should be in CET time, like '17:05 CET'.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:734", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:734", "question": "When did the number of tweets referencing Germanwings exceed 700,000?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "yes", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:735", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:735", "question": "Is the Germanwings Facebook account logo consistent before and after the first 120 minutes?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:736", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:736", "question": "In how many hours Airbus incorporated a pop-up notification acknowledging the incident?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "18", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 18}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:737", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:737", "question": "How many blogs mentioned Germanwings?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:738", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:738", "question": "How many blogs mentioned Airbus?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:739", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:739", "question": "How many instances of Lufthansa's official website are presented?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['2015-03-24', '2015-03-25']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}, {"region": {"page": 2}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:740", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:740", "question": "What are the incident and report dates? Save your answer as a list, with each data formatted as 'YYYY-MM-DD'.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "41.67", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:germanwingsdigitalcrisisanalysis-150403064828-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:741", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:741", "question": "How many percent of Germanwings focused tweets are in English?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:742", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:742", "question": "How many percent of Airbus focused tweets are in English?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['ENGINE TEMP', 'INTAKE AIR TEMP', 'MASS AIR FLOW', 'THROTTLE POSITION', 'HEGO', 'CRANKSHAFT', 'CAMSHAFT']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:743", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:743", "question": "Which seven sensors are connected to the ECU?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Single Point Fuel Injection", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:744", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:744", "question": "In which system the throttle valve is placed beneath the fuel injector?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:745", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:745", "question": "In which system the throttle valve is not required?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "14", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 23}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 24}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 29}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:746", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:746", "question": "How many figures are presented in the section of sensors?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:748", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:748", "question": "How many figures illustrated the structure of fuel injectors?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Engine temperature sensor', 'Inlet air temperature sensor']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 25}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:749", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:749", "question": "What temperature sensors are illustrated with figures in the slides?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['cold start system', 'fuel metering system', 'fuel pump']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}, {"region": {"page": 32}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:750", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:750", "question": "What are connected with fuel supply?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 15}, "source_id": "mmlongbench:efis-140411041451-phpapp01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:751", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:751", "question": "How many outputs does the example ECU have?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:752", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:752", "question": "How many outputs does the example EFI have?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "egocentric planning", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2306.05425v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:753", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:753", "question": "What task is on the top of the planning branch of the tree?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:754", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:754", "question": "What task is on the top of the action branch of the tree?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:2306.05425v1.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:2306.05425v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:755", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:755", "question": "How many bar plots (including subplots) are in this paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Prompt", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:2306.05425v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:756", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:756", "question": "According to the figure, what is the first step of cold start?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "soccer", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2306.05425v1.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:2306.05425v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:757", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:757", "question": "What sport is shown in the visual examples of MIMIC-IT and Otter?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:758", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:758", "question": "What literature is shown in the visual examples of MIMIC-IT and Otter?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:2306.05425v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:759", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:759", "question": "How many languages are shown in the in-context examples for multi-lingual translation query?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:761", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:761", "question": "How many line plots are shown in the slides?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2210.02442v1.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:2210.02442v1.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:2210.02442v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:762", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:762", "question": "How many figures convey similar messages as the first figure?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:2210.02442v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:763", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:763", "question": "How many strategies outperform the random selection baseline on CIFAR-10-LT?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:764", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:764", "question": "How many strategies outperform the uniform sampling baseline on CIFAR-10-LT?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "VAAL", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2210.02442v1.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:2210.02442v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:765", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:765", "question": "What querying criteria yields the worst label diversity generally on all the datasets?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:2210.02442v1.pdf"}, {"region": {"page": 24}, "source_id": "mmlongbench:2210.02442v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:766", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:766", "question": "How many line plots are in this paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "SGD", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:2210.02442v1.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:2210.02442v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:767", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:767", "question": "What are the optimizers used in this research?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:768", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:768", "question": "What are the large language models used in this research?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "ground truth", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:2210.02442v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:769", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:769", "question": "On the dataset map of PathMNIST, does the map by ground truth or pseudo-labels have a larger confidence variance?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:770", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:770", "question": "On the dataset map of DermaMNIST, does the map by ground truth or pseudo-labels have a larger confidence variance?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['Figure 5', 'Figure 6']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:2405.09818v1.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:2405.09818v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:771", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:771", "question": "Which figures include line plots in the paper?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "text tokens", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2405.09818v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:772", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:772", "question": "What do the green squares denote in Fig 1?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:773", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:773", "question": "What do the red squares denote in Fig 1?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "18", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2405.09818v1.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:2405.09818v1.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:2405.09818v1.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:2405.09818v1.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:2405.09818v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:774", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:774", "question": "How many pictures are used in the chat example figures, excluding the Appendix?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:2405.09818v1.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:2405.09818v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:775", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:775", "question": "How many subplots are related to obvious loss spikies?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "no", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:2405.09818v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:776", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:776", "question": "Does llama-1 apply qknorm? Directly answer 'yse' or 'no'", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:777", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:777", "question": "Does llama-3 apply qknorm?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "150k", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:2405.09818v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:778", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:778", "question": "At which step, the loss spikes when without QK-norm?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "panda", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:779", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:779", "question": "What is the first animal, other than humans, shown in this guidebook?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:780", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:780", "question": "What is the coldest month in Sichuan?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 45}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 46}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 47}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 48}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 49}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 50}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 51}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 52}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 53}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 54}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 55}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 56}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:781", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:781", "question": "How many figures are presented in the infrastructure section?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "green", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:782", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:782", "question": "What color are the birds on the sides of the golden sunbird disc design?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1217.02", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:783", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:783", "question": "What is Chendu's latest GDP in billion Rmb?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 26}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:784", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:784", "question": "Which line's gates is shown as an example of the train map?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Chen Mapo Doufu', 'Yu Zhi Lan', 'Wenshu Monastery Teahouse']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}, {"region": {"page": 39}, "source_id": "mmlongbench:Sinopolis-Chengdu.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:785", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:785", "question": "What restaurants/teahouses in Qingyang District are recommended? Answer in English.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "ILF0000808", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:BRO-GL-MMONEY.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:795", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:795", "question": "What is the Account No. of Account 7 in the portfolio overview example?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:796", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:796", "question": "What is the Account No. of Account 6 in the portfolio overview example?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['US Dollar', 'Euro', 'Pound']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:BRO-GL-MMONEY.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:797", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:797", "question": "What are the currencies shown in Multiple settlement options?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:BRO-GL-MMONEY.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:BRO-GL-MMONEY.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:798", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:798", "question": "How many images are shown in the section of trading capabilities?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Live Balances", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:BRO-GL-MMONEY.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:799", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:799", "question": "What is the button to click for up-to-the-minute portfolio balances?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "pie chart", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:BRO-GL-MMONEY.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:800", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:800", "question": "Which type of chart is shown in the what-if analysis?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Traceability', 'Data Lake', 'Integration', 'Security', 'Orchestration']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:801", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:801", "question": "What are the services of ValueEdge ops?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "10", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:802", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:802", "question": "How many faces are shown in this document?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "ValueEdge insights", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:803", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:803", "question": "What is the title of the page that contains a screenshot?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:804", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:804", "question": "What is the title of the page that contains a line plot?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "default_workspace", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:805", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:805", "question": "What is the name of the workspace shown as an example?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Plan', 'Build', 'Test', 'Deliver', 'Run']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:transform-software-delivery-with-valueedge-brochure.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:806", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:806", "question": "What are the five steps of ValueEdge Insights?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "61%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 36}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:807", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:807", "question": "In the survey conducted May 1-15, 2017, what percentage of U.S. adults says the number of people killed or injured in traffic accidents will not decrease if driverless vehicles become widespread?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:808", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:808", "question": "In the survey conducted May 1-15, 2023, what percentage of U.S. adults who say the number of people killed or injured in traffic accidents will decrease if driverless vehicles become widespread?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:809", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:809", "question": "For those who are Phd students, what percentage of U.S. workers say technology generally makes their jobs more interesting", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "36%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:810", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:810", "question": "What percentage of U.S. adults who say it will take more than 50 years for most vehicles on the road to be driverless?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2481", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:811", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:811", "question": "How many US workers say email or social media have had a postive impact on their own careers or jobs?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "9", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:812", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:812", "question": "How many chart can be found in the first 10 pages of the report?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1695", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}, {"region": {"page": 42}, "source_id": "mmlongbench:PI_2017.10.04_Automation_FINAL.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:813", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:813", "question": "How many US workers are interested in a robot caregiver for themselves or a family member?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Hamilton', 'Lucas', 'Murphy', 'York', 'Aurora']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:822", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:822", "question": "What are the counties mentioned in the document?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "538", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:823", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:823", "question": "How many square miles does the Hamilton country covers according to the document? Give me an appropriate integer.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}, {"region": {"page": 14}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:824", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:824", "question": "how many tables are included in the document?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "150", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:698bba535087fa9a7f9009e172a7f763.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:825", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:825", "question": "For the city where the first farmers' elevator was located, what's the Population there in 2000?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Microsoft Office OneNote', 'SOAPWare']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:826", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:826", "question": "Which application software interfaces are displayed in the slides?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Electronic Medical Records", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:827", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:827", "question": "Among the most important applications listed, which one is shown a picture of the interface as an example", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 20}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:828", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:828", "question": "How many slides include more than one figure? Here don't consider the charts.", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:829", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:829", "question": "How many lines are highlighted in yellow on the first page? Give me an integer.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:830", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:830", "question": "When will the market open?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Use the Google search engine', 'Pick a small business in your community', 'Pick a specific product that you use frequently']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:831", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:831", "question": "How to prepare for Tomorrow's Workplace Skills for QUARTER 3?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Understanding Economic Systems and Business', 'Making Ethical Decisions and Managing a Socially Responsible Business']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}, {"region": {"page": 2}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:832", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:832", "question": "what are the names of Unit 1 and Unit 2 in Semester 1:quarter 1?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Discrimination in the Workplace Continues", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:833", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:833", "question": "What is the title of case study given under unit-8 in table?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:834", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:834", "question": "How many critical thinking case studies are included in all the assignments?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "10", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:835", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:835", "question": "how many LEARNING OUTCOMES should be ANSWERed in UNIT 8?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Using Financial Information and Accounting", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:f8d3a162ab9507e021d83dd109118b60.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:836", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:836", "question": "what's the topic of UNIT 14?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:837", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:837", "question": "What is the difference of listening time in Avg. weekly hours between 2013 and 2014?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:838", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:838", "question": "What is the difference of listening time in Avg. weekly Hours between 2013 and the year when Streaming had a 20% share of the business?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "23", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:839", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:839", "question": "What is the combined percentage of Album Sales % and Song Sales % for the Country genre?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Physical albums', 'Digital albums', 'Digital tracks']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:840", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:840", "question": "What kind of albums are reducing the share of their business due to streaming?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1194000", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 14}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:841", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:841", "question": "What is the difference in total volume between the rank 1 and rank 19 top albums?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:842", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:842", "question": "What is the difference in total volume between the rank 1 and rank 17 top albums?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 30}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 31}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 32}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 33}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 34}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 35}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:843", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:843", "question": "How many exmaples are shown to support the importance of seeing the whole picture?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "7", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 30}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 31}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 32}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 33}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 34}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}, {"region": {"page": 35}, "source_id": "mmlongbench:nielsen2015musicbizpresentation-final-150526143534-lva1-app6891_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:844", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:844", "question": "How many distinct people are involved in the pictures in Pages 30-35?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:845", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:845", "question": "whose album is ranked 8 according to song sales in 2024 on billboard?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['Increasing world population', 'Increasing energy demand', 'Limited energy supplies', 'Environmental effects of energy use']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:846", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:846", "question": "What are the four concrect facts of global challenges?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:847", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:847", "question": "What are the ten concrect consequences of global challenges?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['World Mobility', 'World CO2 Emission', 'World Population']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:848", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:848", "question": "Among the global challenges and requirements, which of them would be constantly increasing between 1950 and 2050?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:849", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:849", "question": "Among the global challenges, which of them would not be constantly increasing between 2050 and 2100?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "['Nitrogen', 'Water', 'Carbon Dioxide', 'Pollutants']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:850", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:850", "question": "What four elements are exhausted in the combustion process of vehicles?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Power Generation', 'Vehicle Traffic', 'Industry']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 16}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:851", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:851", "question": "What are the top 3 sources according to the total emission in percent by weight? List them in a descending order", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Cough', 'Phlegm', 'Adult Resp.']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:852", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:852", "question": "List all the PM health effects that increse by more than 35% in India and Thailand.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['13.7%', '0.1%']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 48}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 49}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:853", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:853", "question": "For first two gases that causes greenhouse effect, list their proportions of the exhaust gas from Gasoline-engine.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['USA', 'Russia', 'South Korea']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 55}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 58}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:854", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:854", "question": "One kind of gas is the siginificant contributor to climate change. What are the top 3 countries that produces this gas most?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "24", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 25}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 27}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 48}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 53}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 56}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 57}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 58}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 59}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 61}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 62}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 63}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 66}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 67}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 68}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 70}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}, {"region": {"page": 75}, "source_id": "mmlongbench:dr-vorapptchapter1emissionsources-121120210508-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:855", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:855", "question": "How many slides includes at least one chart?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "17", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:865", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:865", "question": "How many different icon are shown as different image types in Figure 1?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:866", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:866", "question": "How many different icon are shown as different Disciplines in Figure 10 ?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:867", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:867", "question": "How many charts are included in Pages 5-10?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:868", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:868", "question": "How many tables are included in Pages 105-110?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "\"MMMU\"", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:869", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:869", "question": "According to this paper, among nice different datasets in Fig 4, which one can reflect the most breadth of knowledge?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:870", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:870", "question": "According to this paper, among nice different datasets in Fig 4, which one can reflect the highest efficiency?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Perceptual Error', 'Lack of Knowledge', 'Reasoning Error']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:871", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:871", "question": "List the top-3 error types over 150 annotated GPT-4V errors in Figure 6", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Reasoning Error', 'Perceptual Error', 'Lack of Knowledge']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 96}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 98}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 99}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 100}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:872", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:872", "question": "List all the error types mentioned in the case studies in Pages 95-100", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 65}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 66}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 67}, "source_id": "mmlongbench:2311.16502v3.pdf"}, {"region": {"page": 70}, "source_id": "mmlongbench:2311.16502v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:873", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:873", "question": "How many cases are successful in Pages 65-70?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['IA Co-source', 'IA outsource', 'IA Tech Audit', 'IA Advisory']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:PWC_opportunity_of_lifetime.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:915", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:915", "question": "List the areas that PWC performs IA support across.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1914", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:PWC_opportunity_of_lifetime.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:916", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:916", "question": "How many employees dose PWC Assurance have?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:917", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:917", "question": "How many offices dose PWC Technic have?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "5100", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:PWC_opportunity_of_lifetime.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:PWC_opportunity_of_lifetime.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:PWC_opportunity_of_lifetime.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:PWC_opportunity_of_lifetime.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:918", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:918", "question": "How many employees dose PWC have including Assurance, Consulting, Deals and Tax & Legal Services?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Core Assurance", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:PWC_opportunity_of_lifetime.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:919", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:919", "question": "Who is Ibrahim?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:920", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:920", "question": "Who is Amy?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:PWC_opportunity_of_lifetime.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:921", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:921", "question": "How many ways to connect with pwc people?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "4%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:PP_2021.04.22_voting-access_REPORT.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:922", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:922", "question": "Compared to October 2018, by how much did the proportion of Democrats who support automatically registering all eligible citizens to vote increase by April 2021?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:923", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:923", "question": "Compared to October 2018,  the proportion of Democrats who support automatically registering all eligible citizens to vote decreases.", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "14%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:PP_2021.04.22_voting-access_REPORT.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:924", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:924", "question": "What percentage of Republicans strongly favor automatically registering all eligible citizens to vote?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "18-34", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:PP_2021.04.22_voting-access_REPORT.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:925", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:925", "question": "Among Republicans, which age group has the highest proportion of support for making Election Day a national holiday?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "15", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:PG_20.07.30_U.S.-Views-China_final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:926", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:926", "question": "What percentage does Republicans in the United States rate China's response good to the COVID-19 pandemic?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "21%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:PG_20.07.30_U.S.-Views-China_final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:927", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:927", "question": "How much did the proportion of the favorable views of China among the American public decresed from 2005 to 2020?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:928", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:928", "question": "How much did the proportion of the favorable views of China among the American public incresed from 2005 to 2020?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "54%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 5}, "source_id": "mmlongbench:PG_20.07.30_U.S.-Views-China_final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:929", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:929", "question": "Assume that in a survey conducted after July 2020, the percentage of Americans who believe China has done a \"good\" job dealing with the coronavirus outbreak increased by 10 percentage points, then what percentage of Americans would believe China has done a \"bad\" job (assuming the percentage of all the other options stays the same)?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "29%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:PG_20.07.30_U.S.-Views-China_final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:930", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:930", "question": "Assume that after July 2020, the percentage of Republicans who believe \"China is an enemy of the U.S.\" increased by 10 percentage points, while the percentage of Democrats remained the same.  What would be the difference in the percentage between Republicans and Democrats at that time?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "50 and older", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:PG_20.07.30_U.S.-Views-China_final.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:931", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:931", "question": "which age group experienced the greatest change in the percentage holding an \"unfavorable\" opinion of China between 2005 and 2010?", "scope": {"holder": "Research-report-Introduction", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Medical Informatics and HIT Systems", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:932", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:932", "question": "What is the title of the slide?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "David Lubliner", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:933", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:933", "question": "who is the presenter of this slide?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "10", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 9}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 10}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 12}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 13}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 15}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 17}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:934", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:934", "question": "How many lateral bar graphs are present in the document?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "[2005, 2006]", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 12}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:935", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:935", "question": "TOP security concerns of computerized medical information DIAGRAM DISCUSSED ABOUT WHICH YEAR? List all of them like ['2014', '2018', '2020'].", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['Mircosoft Office OneNote', 'Windows Journal', 'Microsoft Educational Pack for Tablet PC', 'Microsoft Experience Pack for Tablet PC', 'Sticky Notes', 'Snipping Tool']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 6}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:936", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:936", "question": "Mention Tablet Capture Devices used by some hospitals?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Patient registration/ demographics', 'Insurance validation', 'Billing systems', 'Appointment systems', 'Computerized Physician Order Entry (CPOE)', 'EMR/EHR/CPR', 'Pharmacy systems', 'Lab systems', 'Imaging systems', 'Telemedicine', 'Patient education', 'Decision Support', 'Library resources', 'Sensors']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:937", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:937", "question": "What are the HIT applications?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:c31e6580d0175ab3f9d99d1ff0bfa000.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:938", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:938", "question": "How many times does \"HIT\" appear in slide?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "30216492.00", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:f86d073b0d735ac873a65d906ba82758.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:939", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:939", "question": "What is total dividend of Erstwhile ITC Hotels Limited (in Rupee) between 1999 to 2000 year?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:940", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:940", "question": "What is total dividend of Erstwhile ITC Hotels Limited (in Rupee) between 2002 to 2003 year?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "Blue", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 0}, "source_id": "mmlongbench:f86d073b0d735ac873a65d906ba82758.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:941", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:941", "question": "WHAT IS ITC LIMITED LOGO COLOR?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "44.96%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:f86d073b0d735ac873a65d906ba82758.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:943", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:943", "question": "What percentage of the shareholder was held by foreign companies and institutional investors as of March 31, 2007?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:944", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:944", "question": "What percentage of the shareholder was held by foreign companies and institutional investors as of March 31, 2017?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:f86d073b0d735ac873a65d906ba82758.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:f86d073b0d735ac873a65d906ba82758.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:945", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:945", "question": "Among all in-office board members,  how many other chairmanships the person who attended the least board meetings hold on the boards of other companies?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "51.02%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:f86d073b0d735ac873a65d906ba82758.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:946", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:946", "question": "What is the sum ratio of shares held of the top-3 shareholders as on 31st, March, 2007?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:f86d073b0d735ac873a65d906ba82758.pdf"}, {"region": {"page": 7}, "source_id": "mmlongbench:f86d073b0d735ac873a65d906ba82758.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:947", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:947", "question": "What is total committee strengh of audit committee meetings and compensation committee meetings in 21st July,  2006?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "LIBERTARIAN PARTY OF GEORGIA, INC", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:a4f3ced0696009fec3179f493e4f28c4.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:948", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:948", "question": "What company is a plaintiff?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "21-13199", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:a4f3ced0696009fec3179f493e4f28c4.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:949", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:949", "question": "WHAT IS USCA CASE NUMBER?", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2022-01-05", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:a4f3ced0696009fec3179f493e4f28c4.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:950", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:950", "question": "What is the date of the document? Format your answer as YYYY-MM-DD.", "scope": {"holder": "Administration-Industry-file", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:951", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:951", "question": "From 2009 to 2013, as for the adviser's organic growth rate,  how many years are higher than 2011?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:952", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:952", "question": "From 2009 to 2013, as for the adviser's organic growth rate,  how many years are higher than 2010?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "11", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:953", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:953", "question": "How many years have there been more than 4,000 births (thousands)?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:954", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:954", "question": "How many years have there been more than 4,500 births (thousands)?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['line them up', 'observe your clients', 'build a buyer persons']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 37}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}, {"region": {"page": 39}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}, {"region": {"page": 41}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:955", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:955", "question": "In the \"Who are you talking to\" section,  what tips does the author give us?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "18", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}, {"region": {"page": 19}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:956", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:956", "question": "On what page does the first part end?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['slideshare', 'twitter', 'linkedin', 'podcast']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 21}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}, {"region": {"page": 62}, "source_id": "mmlongbench:csewt7zsecmmbzjufbyx-signature-24d91a254426c21c3079384270e1f138dc43a271cfe15d6d520d68205855b2a3-poli-150306115347-conversion-gate01_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:957", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:957", "question": "what are the overlapped apps between page 21 and 62?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['shiny object', 'mirror']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:b3m5kaeqm2w8n4bwcesw-140602121350-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:958", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:958", "question": "What are two examples of reflecting surfaces?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['radiowaves', 'microwaves']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 17}, "source_id": "mmlongbench:b3m5kaeqm2w8n4bwcesw-140602121350-phpapp02_95.pdf"}, {"region": {"page": 18}, "source_id": "mmlongbench:b3m5kaeqm2w8n4bwcesw-140602121350-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:959", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:959", "question": "What are used for long-distance communication as introduced in this slide?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Yes", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 18}, "source_id": "mmlongbench:b3m5kaeqm2w8n4bwcesw-140602121350-phpapp02_95.pdf"}, {"region": {"page": 20}, "source_id": "mmlongbench:b3m5kaeqm2w8n4bwcesw-140602121350-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:960", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:960", "question": "Does CFCs causes skin burn? Directly answer 'yes' or 'no'.", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "the chemical reaction involving CFCs.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 21}, "source_id": "mmlongbench:b3m5kaeqm2w8n4bwcesw-140602121350-phpapp02_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:961", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:961", "question": "Cold is the catalyst for what reaction?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:disciplined-agile-business-analysis-160218012713_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:962", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:962", "question": "How many black arrows are part of the DISCIPLINED AGILE logo?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:963", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:963", "question": "How many blue arrows are part of the DISCIPLINED AGILE logo?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:disciplined-agile-business-analysis-160218012713_95.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:disciplined-agile-business-analysis-160218012713_95.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:disciplined-agile-business-analysis-160218012713_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:964", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:964", "question": "How many different twitter accounts are shown in slide?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:disciplined-agile-business-analysis-160218012713_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:965", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:965", "question": "How many different Chinese characters are shown in slide?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:966", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:966", "question": "How many different Japanese characters are shown in slide?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "5", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:disciplined-agile-business-analysis-160218012713_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:967", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:967", "question": "How many times does \"Barclays\" appear on page 4?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['concept', 'production']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 10}, "source_id": "mmlongbench:disciplined-agile-business-analysis-160218012713_95.pdf"}, {"region": {"page": 11}, "source_id": "mmlongbench:disciplined-agile-business-analysis-160218012713_95.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:968", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:968", "question": "What stages does high level lifecycle have more than basic lifecycle?", "scope": {"holder": "Tutorial-Workshop", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['event relation extraction', 'deductive reasoning']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:2310.09158v1.pdf"}, {"region": {"page": 2}, "source_id": "mmlongbench:2310.09158v1.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:2310.09158v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:969", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:969", "question": "The author mainly conducted experiments on which two tasks?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:2310.09158v1.pdf"}, {"region": {"page": 2}, "source_id": "mmlongbench:2310.09158v1.pdf"}, {"region": {"page": 3}, "source_id": "mmlongbench:2310.09158v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:970", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:970", "question": "According to the definition in the paper, how many possible combinations of relationships can there be between two events?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:971", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:971", "question": "According to the paper, there are five different relation types between two events.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:972", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:972", "question": "The authors propose four different solutions for logical reasoning of LLMs.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "CoT w. logical constraints", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:2310.09158v1.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:973", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:973", "question": "Which baseline did the pre-trained Vicuna-13B perform best on MAVEN-ERE?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2312.04350v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:974", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:974", "question": "how many key steps did the author divide causal inference into?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:975", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:975", "question": "the author divide causal inference into 5 key steps.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "semantic parsing", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 2}, "source_id": "mmlongbench:2312.04350v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:976", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:976", "question": "What subskill does we need to collect the available data?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "intervention", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2312.04350v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:977", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:977", "question": "What rung is the second ladder of causation refer to?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:978", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:978", "question": "There are four different rungs for the ladder of causation", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "2", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2312.04350v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:979", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:979", "question": "According to the definition in the paper, how many directed edges are there in a chain-type causal graph?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "GPT-4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:2312.04350v3.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:980", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:980", "question": "Which model performs the best on Cladder?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "No", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2305.14160v4.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:2305.14160v4.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:981", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:981", "question": "Does the mean significance of information flow from the text part to label words always greater than the mean significance of information flow from label words to the target position? Answer 'yes' or 'no' directly.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "No", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:2305.14160v4.pdf"}, {"region": {"page": 4}, "source_id": "mmlongbench:2305.14160v4.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:982", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:982", "question": "Does the  information flow from label words to the target position dominate in all layers? Answer 'yes' or 'no' directly.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Yes", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:2305.14160v4.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:983", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:983", "question": "Does randomly isolating non-label words within the last 5 layers have almost no impact? Answer 'yes' or 'no' directly.", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "16.74", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:2305.14160v4.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:984", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:984", "question": "How much does the Anchor Re-weighting method improve the performance of vanilla ICL on average?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Description", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 8}, "source_id": "mmlongbench:2305.14160v4.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:986", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:986", "question": "What is the most confusing category of Abbreviation?", "scope": {"holder": "Academic-paper", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "5:00pm", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:SAO-StudentSupport_Guidebook-Content.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:987", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:987", "question": "Excepted for the public holidays, when is the counter closed on Tuesday according to the provided operating hours?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "has-exch@ntu.edu.sg", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:SAO-StudentSupport_Guidebook-Content.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:988", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:988", "question": "Which email should an exchange student contact for housing matters?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "(65) 6790 8331", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:SAO-StudentSupport_Guidebook-Content.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:989", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:989", "question": "What is thed dental telephone number for fullerton healthcare at NTU?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Campus Security", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 25}, "source_id": "mmlongbench:SAO-StudentSupport_Guidebook-Content.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:990", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:990", "question": "When a student cannot be found, who should we contact?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "4", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 33}, "source_id": "mmlongbench:SAO-StudentSupport_Guidebook-Content.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:991", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:991", "question": "At NTU, how many types of Field Sports can students participate in?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "Nottingham Trent University", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 1}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:992", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:992", "question": "What is NTU standing for in this guidebook?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:993", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:993", "question": "How many social media accounts does NTU have?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}, {"region": {"page": 62}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}, {"region": {"page": 63}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}, {"region": {"page": 64}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}, {"region": {"page": 65}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}, {"region": {"page": 66}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:994", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:994", "question": "Starting from page 62, how many pages are there about the introduction of the scholarship?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "88%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:995", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:995", "question": "How many students of NTU would recommend studying at NTU?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "NTIC", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 33}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:996", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:996", "question": "Where did Terry Tan study the foundation course before he progressed to NTU?", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "15 mins", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 61}, "source_id": "mmlongbench:Guide-for-international-students-web.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:997", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:997", "question": "How much time does it take from clifton campus to the city centre by bus? Your answer should be formatted as 'xx mins'.", "scope": {"holder": "Guidebook", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "BBA - Bachelor of Business Administration", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:NUS-Business-School-BBA-Brochure-2024.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:998", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:998", "question": "From 2022 graduate employment survey,  graduates with which degree have the highest average monthly salary?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:999", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:999", "question": "From 2022 graduate employment survey,  do graduates with BBA (Hon) degree have the highest average monthly salary?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": true}
{"answer": "['Land Law', 'Urban Economics', 'Real Estate Investment Analysis', 'Urban Planning', 'Principles of Real Estate Valuation', 'REIT and Business Trust Management']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:NUS-Business-School-BBA-Brochure-2024.pdf"}, {"region": {"page": 8}, "source_id": "mmlongbench:NUS-Business-School-BBA-Brochure-2024.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1000", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1000", "question": "What are the representative courses for majors that need 64 major curriculum units?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 24}, "source_id": "mmlongbench:NUS-Business-School-BBA-Brochure-2024.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1001", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1001", "question": "How many social media accounts might NUS Business School have?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "3", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:NUS-Business-School-BBA-Brochure-2024.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1002", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1002", "question": "In the photo provided by BRYAN VOON,  how many different people are there wearing glasses?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "16", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:NUS-Business-School-BBA-Brochure-2024.pdf"}, {"region": {"page": 16}, "source_id": "mmlongbench:NUS-Business-School-BBA-Brochure-2024.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1003", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1003", "question": "How many weeks do students need to reach work experience milestone and get 8 units?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 6}, "source_id": "mmlongbench:2021-Apple-Catalog.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1004", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1004", "question": "How many different iPad solutions are shown in the document?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": null, "answer_format": "none", "as_of": "2025-11-06T05:23:19+00:00", "gold": [], "modality": "pdf", "origin": {"construction": "native unanswerable", "dataset": "yubo2333/MMLongBench-Doc", "item": "row:1005", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1005", "question": "Are there 11 different iPad solutions shown in the document?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": true}
{"answer": "4", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 22}, "source_id": "mmlongbench:2021-Apple-Catalog.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1006", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1006", "question": "Regarding macbook high security solution, which numbers of features are not presented in the illustration?", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Yes", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 11}, "source_id": "mmlongbench:2021-Apple-Catalog.pdf"}, {"region": {"page": 28}, "source_id": "mmlongbench:2021-Apple-Catalog.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1007", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1007", "question": "One40 can only be used for Apple Watch, is that true? Please directly answer 'yes' or 'no'.", "scope": {"holder": "Brochure", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "24.26", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 69}, "source_id": "mmlongbench:ACTIVISIONBLIZZARD_2019_10K.pdf"}, {"region": {"page": 70}, "source_id": "mmlongbench:ACTIVISIONBLIZZARD_2019_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1008", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1008", "question": "what is the FY2019 fixed asset turnover ratio for Activision Blizzard?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1.9%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 70}, "source_id": "mmlongbench:ACTIVISIONBLIZZARD_2019_10K.pdf"}, {"region": {"page": 73}, "source_id": "mmlongbench:ACTIVISIONBLIZZARD_2019_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1009", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1009", "question": "What is the FY2017 - FY2019 3 year average of capex to revenue ratio for Activision Blizzard?Answer in units of percents and round to one decimal place.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "5.03%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 47}, "source_id": "mmlongbench:ACTIVISIONBLIZZARD_2019_10K.pdf"}, {"region": {"page": 69}, "source_id": "mmlongbench:ACTIVISIONBLIZZARD_2019_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1010", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1010", "question": "what is R&D to asset ratio for Activision Blizzard in FY 2019?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "17.85", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 70}, "source_id": "mmlongbench:ACTIVISIONBLIZZARD_2019_10K.pdf"}, {"region": {"page": 114}, "source_id": "mmlongbench:ACTIVISIONBLIZZARD_2019_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1011", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1011", "question": "what is Interest Coverage Ratio for Activsion Blizzard In F2019?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:ACTIVISIONBLIZZARD_2019_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1012", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1012", "question": "what value stock does Activsion Blizzard repurchase in FY2019?Answer in USD billion", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "18078", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1013", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1013", "question": "What is common equity  for COSTCO in FY2021?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "11749", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1014", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1014", "question": "What is total long-term Liabilities for COSTCO in FY 2021? Anwser in millions", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8489", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 36}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1015", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1015", "question": "what is EBITDA  for costco in FY2021?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "10314", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 53}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1016", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1016", "question": "what is long-term debt of  Costco in FY 2021? Anwser in millions.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "11407", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 53}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1017", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1017", "question": "what is total debt of COSTCO in FY 2021?Answer in millions.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1.344", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 36}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 53}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1018", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1018", "question": "what is total debt to EBITDA ratio of COSTCO in FY2021?round your answer to three decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "41190", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1019", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1019", "question": "what is total liabilities for COSTCO in FY 2021?Answer in millions.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.25", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 53}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1020", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1020", "question": "what is Long-term Debt to Total Liabilities for COSTCO in FY2021? Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.715", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1021", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1021", "question": "what is current liabilities to total liabilities for COSTCO in FY2021? Round your answer to three decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "41190", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 36}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1022", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1022", "question": "What is total liabilities for costco in FY 2021?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.192", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 53}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1023", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1023", "question": "what is total debt to total assets  for costco in FY 2021?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.304", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1024", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1024", "question": "what is Operating cash flow to Current Liabilitiesfor COSTCO in FY2021? Round your answer to three decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1358000", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1025", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1025", "question": "How much shares did COSTCO repurchase in FY2021?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "495.00", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 29}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1026", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1026", "question": "How much did COSTCO spend on repurchasing of common stock in FY2021?Answer in millions.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "the financial performance of our U.S. and Canadian operations.", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 9}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1027", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1027", "question": "What does Costco rely heavily on for its financial performance in FY2021?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "815", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:COSTCO_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1028", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1028", "question": "How many membership warehouses did COSTCO operate at 2021?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "5.4%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}, {"region": {"page": 42}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1029", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1029", "question": "what is the FY2015 unadjusted EBITDA % margin for Netflix?Answer in percentage term, round to one decimal place.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1902216.00", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 43}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1030", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1030", "question": "What is Netflix working capital in FY2015?Answer in thousands.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3.56", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}, {"region": {"page": 43}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1031", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1031", "question": "What is sales to working capital ratio of Neflix in FY 2015?Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "253.3", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 24}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1032", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1032", "question": "What amount did  personnel-related costs increase for Netfilx in 2015? Answer in millions.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "714.3", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 47}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1033", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1033", "question": "what is advertsing expense of Neflix in FY 2015? Answer in millions", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.105", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}, {"region": {"page": 47}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1034", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1034", "question": "what is advertsing expense to sales ratio of Neflix in FY 2015? Round your answer to three decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "650788.00", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 24}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1035", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1035", "question": "what is research and development expense of Neflix in FY 2015? Answer in thousands.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3.049", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}, {"region": {"page": 43}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1036", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1036", "question": "what is sales to stockholder equity ratio  of Neflix in FY 2015? Round your answer to three decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "seven-for-one stock split", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 55}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1037", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1037", "question": "what method did netflix use to pay the dividend to shareholders in FY2015.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "34.7", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 54}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1038", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1038", "question": "what is operating leases occurred in FY 2015 for Netfilx?Answer in million.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "30.3%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1039", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1039", "question": "what percentage did Global streaming memberships increase from 2014 to 2015?Answer in percentage term, round to one decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1375500.00", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 21}, "source_id": "mmlongbench:NETFLIX_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1040", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1040", "question": "what profit  did Memberships contribute to in Domestic Streaming Segment in FY2015? Answer in thousands.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3.46", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 59}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}, {"region": {"page": 61}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1041", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1041", "question": "what is the FY2021 inventory turnover ratio for Nike?Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "12.35", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 59}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}, {"region": {"page": 61}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1042", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1042", "question": "What is receive turnover in FY2021 for Nike? Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "9.67", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 59}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}, {"region": {"page": 61}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1043", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1043", "question": "What is payables  turnover in FY2021 for Nike? Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1.02", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 61}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1044", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1044", "question": "What is  cash_ratio in FY2021 for Nike? Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "97.40", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 59}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}, {"region": {"page": 61}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1045", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1045", "question": "What is  cash conversion cycle in FY2021 for Nike? Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "2.72", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 61}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1046", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1046", "question": "What is current ratio in FY2021 for Nike?  Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1.02", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 61}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1047", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1047", "question": "What is cash ratio  in FY2021 for Nike? Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1.85", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 61}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1048", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1048", "question": "What is quick ratio cycle in FY2021 for Nike? Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "6063.00", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 72}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1049", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1049", "question": "what is accrual liabilities in FY2021 for Nike? Answer in millions.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Running', 'Nike Basketball', 'the Jordan Brand', 'Football (Soccer)', 'Training', 'Sportswear (sports-inspired lifestyle products)']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 3}, "source_id": "mmlongbench:NIKE_2021_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1050", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1050", "question": "What are the  key categories that Nike focuses on for their brand product offerings?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "97.75", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1051", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1051", "question": "What is Amazon's FY2017 days payable outstanding (DPO)?Round your answer to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.073", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}, {"region": {"page": 71}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1052", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1052", "question": "what is Amazon's FY2017 Operating Profit Margin Before Depreciation? round your answer to three decimal", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.023", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 35}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}, {"region": {"page": 38}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1053", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1053", "question": "what is Amazon's FY2017 return on asset ? round your answer to three decimal", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.109", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1054", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1054", "question": "what is Amazon's FY2017 return on equity? round your answer to three decimal", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.202", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}, {"region": {"page": 64}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1055", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1055", "question": "what is Amazon's FY2017 effective tax rate? round your answer to three decimal", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.052", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1056", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1056", "question": "what is Amazon's FY2017 Interest Expense to Average Long-Term Debt Ratio?  round your answer to three decimal", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.198", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}, {"region": {"page": 50}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1057", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1057", "question": "what is Amazon's FY2017 cash balance to total liability ratio? round your answer to three decimal", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1.93", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 38}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}, {"region": {"page": 56}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}, {"region": {"page": 71}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1058", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1058", "question": "what is Amazon's FY2017 debt to ebitda ratio? round your answer to three decimal", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "60.3%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 43}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1059", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1059", "question": "what is the percentage change of  return for allowance from 2016 to  2017? Round your answer to one decimal", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "['the purchase price of consumer products', 'digital media content costs', 'packaging supplies', 'sortation and delivery centers and related equipment costs', 'inbound and outbound shipping costs']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 44}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1060", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1060", "question": "what are the components of cost of sales  for Amazon's FY2017?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "straight-line basis without regard to deferred payment terms", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 47}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1061", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1061", "question": "How do Amazon recognize least cost?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3.0", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 48}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1062", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1062", "question": "what amount is liabilities for unredeemed gift cards in FY2017 for Amazon? Answer in billion.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8.8", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 52}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1063", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1063", "question": "what is depreciation expense on property and equipment in 2017 for Amazon? Answer in billion", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "3362", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 55}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1064", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1064", "question": "what is the total amount of  expected future amortization expense of acquired intangible assets as of December 31, 2017 for Amazon? Answer in million", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8438", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 58}, "source_id": "mmlongbench:AMAZON_2017_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1065", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1065", "question": "what is the total long-term capital lease obligation related to technology infrastructure, and buildings? Answer in million", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.66", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 59}, "source_id": "mmlongbench:ADOBE_2015_10K.pdf"}, {"region": {"page": 63}, "source_id": "mmlongbench:ADOBE_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1066", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1066", "question": "what is the FY2015 operating cash flow ratio for Adobe?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.053", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 59}, "source_id": "mmlongbench:ADOBE_2015_10K.pdf"}, {"region": {"page": 60}, "source_id": "mmlongbench:ADOBE_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1067", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1067", "question": "what is roa for ADBE in FY2015?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.77", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 63}, "source_id": "mmlongbench:ADOBE_2015_10K.pdf"}, {"region": {"page": 99}, "source_id": "mmlongbench:ADOBE_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1068", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1068", "question": "What is Cash Flow to Total Debt Ratio for ADBE In FY2015?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "a software subscription model", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 13}, "source_id": "mmlongbench:ADOBE_2015_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1069", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1069", "question": "what channel  is the majority of ADBE product sold by in FY2015?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1.08%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1070", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1070", "question": "what is the change of  Best Buy's gross margins change from FY2022 to FY2021? Answer in percentage terms, round to 2 decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "Operations activities", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 42}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1071", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1071", "question": "Among operations, investing, and financing activities, which brought in the most (or lost the least) cash flow for Best Buy in FY2022?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "0.263", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 25}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}, {"region": {"page": 39}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1072", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1072", "question": "what is Return on Capital Employed for the fiscal year ending January 28, 2023? round your answer to three decimal places", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.488", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1073", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1073", "question": "what is After-tax Return on Average Equity  for the fiscal year ending January 28, 2023? round your answer to three decimal places", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "2.91%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1074", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1074", "question": "What is Interest to Average Total Debt for the fiscal year ending January 28, 2023? Answer in percentage term, round to three decimal places", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "29.6%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1075", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1075", "question": "what is capitalization ratio for Best Buy for the fiscal year ending January 28, 2023? Answer in percentage term, round to one decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "1874", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 44}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1076", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1076", "question": "what is Cash and cash equivalents for Best Buy for the fiscal year ending January 28, 2023? Answer in million", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "the lower of cost or net realizable value", "answer_format": "str", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 45}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1077", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1077", "question": "how does Best Buy record Merchandise inventories?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "1383", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1078", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1078", "question": "what goodwill does Best Buy have for for the fiscal year ending January 28, 2023?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "0.627", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}, {"region": {"page": 40}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1079", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1079", "question": "what is Gross Profit to Total Assets ratio for Best Buy for the fiscal year ending January 28, 2023? Round Your answer to three decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "25%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 7}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1080", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1080", "question": "what proportion does Best Buy have female employees in new, salaried field positions  for the fiscal year ending January 28, 2023?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "978", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 19}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1081", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1081", "question": "how many Domestic store does Best Buy have in fiscal 2023?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "8802", "answer_format": "int", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 39}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1082", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1082", "question": "what is total current assets in FY2023 for Bestbuy? Answer in million.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "6.31", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1083", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1083", "question": "what is basic earnings per ordinary share in FY2023 for Bestbuy?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "51.286", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1084", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1084", "question": "what is interest coverage ratio for AMCOR'FY 2020? round your answer to three decimal?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "20.7%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1085", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1085", "question": "What is effective tax ratio of Best Buy for for the fiscal year ending January 28, 2023?Answer in percentage term, round to one decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "dev", "unanswerable": false}
{"answer": "13929", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 40}, "source_id": "mmlongbench:BESTBUY_2023_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1086", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1086", "question": "what is invested capital  of Best Buy for for the fiscal year ending January 28, 2023? Answer in million.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "$1577.00", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 60}, "source_id": "mmlongbench:3M_2018_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1087", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1087", "question": "What is the FY2018 capital expenditure amount (in USD millions) for 3M based on cash flow statement?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "['Industrial Business', 'Safety and Graphics Business', 'Health Care Business', 'Electronics and Energy Business', 'Consumer Business']", "answer_format": "list", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 4}, "source_id": "mmlongbench:3M_2018_10K.pdf"}, {"region": {"page": 5}, "source_id": "mmlongbench:3M_2018_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1088", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1088", "question": "what kinds of business or segment does 3M have in FY2018?", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "93", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 104}, "source_id": "mmlongbench:3M_2018_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1089", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1089", "question": "what is the net fair value of derivative instruments for 3M in FY2018? Answer in million.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}
{"answer": "-3.65%", "answer_format": "float", "as_of": "2025-11-06T05:23:19+00:00", "gold": [{"region": {"page": 64}, "source_id": "mmlongbench:3M_2018_10K.pdf"}], "modality": "pdf", "origin": {"dataset": "yubo2333/MMLongBench-Doc", "item": "row:1090", "license": "not-stated-in-card", "revision": "2ff6aa9237fc777b6627dc57a486e9225ac5fb86"}, "query_id": "mmlongbench:1090", "question": "What is the growth rate of advertising and merchandising cost for 3M in FY2018? Answer in percentage term, round to two decimal places.", "scope": {"holder": "Financial-report", "scenario": "document-qa", "session": null}, "split": "test", "unanswerable": false}