---
type: "post"
id: "code-data-and-pre-generated-outputs-available-at-ee47e8"
slug: "code-data-and-pre-generated-outputs-available-at-ee47e8"
title: "Code, data, and pre-generated outputs available at github.com/one-seven17/MER-Bench; accepted to EMNLP 2026."
description: "MER-Bench is a public benchmark for systems that reappraise image-text memes, changing their emotional tone while preserving the scenario and meme style."
retrieval_nugget: "MER-Bench is a public benchmark for systems that reappraise image-text memes, changing their emotional tone while preserving the scenario and meme style."
published_at: "2026-11-03"
updated_at: "2026-11-07"
record_date: "2026-11-03"
date_kind: "published_at"
topics: ["github"]
entities: ["MER-Bench","EMNLP 2026","BAGEL-7B-MoT","DreamOmni2","FLUX.2-klein-4B","Z-Image-Turbo","Anhui University","Hefei University of Technology"]
related_patterns: ["ai-engineering-evaluations-are-splitting-into-task-specific"]
novelty_label: "new_fact"
what_changed: "MER-Bench’s code/data release and EMNLP 2026 acceptance is a new benchmark announcement, distinct from the prior AfrIFact benchmark."
prior_refs: ["https://newruntime.com/posts/dataset-on-huggingface-code-on-github-accepted-at-5c6cec/"]
source_url: "https://github.com/one-seven17/MER-Bench"
source_urls: ["https://github.com/one-seven17/MER-Bench"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:code-data-and-pre-generated-outputs-available-at-ee47e8"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/code-data-and-pre-generated-outputs-available-at-ee47e8/","markdown":"https://newruntime.com/posts/code-data-and-pre-generated-outputs-available-at-ee47e8.md","json":"https://newruntime.com/posts/code-data-and-pre-generated-outputs-available-at-ee47e8.json"}
markdown_url: "https://newruntime.com/posts/code-data-and-pre-generated-outputs-available-at-ee47e8.md"
---

# Code, data, and pre-generated outputs available at github.com/one-seven17/MER-Bench; accepted to EMNLP 2026.

## Retrieval answer

MER-Bench is a public benchmark for systems that reappraise image-text memes, changing their emotional tone while preserving the scenario and meme style.

## Summary
MER-Bench is a public benchmark for systems that reappraise image-text memes, changing their emotional tone while preserving the scenario and meme style. Its repository provides code, dataset access instructions and pre-generated model outputs, alongside evaluation scripts; the paper has been accepted to EMNLP 2026. For builders, it is a reproducible evaluation resource, not evidence that any model has solved the task.

## What happened
The MER-Bench project describes itself as a comprehensive benchmark for multimodal meme reappraisal. The target task is to transform a negative meme into one with a positive, calm or energizing emotional direction while retaining the original scenario and meme style. The authors say the benchmark uses theory-informed emotion targets, paired image-text data and a multi-axis evaluation covering emotion control, content preservation and quality. That framing matters: a system that changes the sentiment but loses the joke, scene or recognizable format would not satisfy the stated task merely by producing a more cheerful image.

The GitHub repository is public under the one-seven17 account. Its README identifies the paper as “MER-Bench: A Comprehensive Benchmark for Multimodal Meme Reappraisal” and lists authors affiliated with Anhui University, Hefei University of Technology, the IAI at Hefei Comprehensive National Science Center, United Arab Emirates University and the University of Science and Technology of China. A repository news entry dated August 2026 says the paper was accepted to EMNLP 2026. The supplied page does not include a publication date for the code release, nor does it give a dataset count or a results summary in the available excerpt.

The project documents a local evaluation workflow. It instructs users to install `uv`, run `uv sync`, and obtain the dataset and pre-generated model outputs with the Hugging Face CLI from `LeoReverse/17nie`. Those files are then organized into `data/`, including original images, an index file and edited outputs grouped by model. The README names example output folders such as BAGEL-7B-MoT, DreamOmni2, FLUX.2-klein-4B and Z-Image-Turbo. The material establishes that artifacts and scripts are provided; it does not establish that every output shown is a complete or independently validated comparison.

For aggregate evaluation, the instructions show a command using `main.py` with ten runs, a sample size of 100, caching and model-view options, followed by a script that writes a CSV from generated JSON. Separate instructions cover factor-level runs across visual type, sentiment polarity and layout, with sample sizes of 50, and then aggregation for Table 3. The page also describes subcategory-level evaluation across models. These are reproduction procedures, not reported performance claims: the supplied source excerpt does not state scores, rankings, uncertainty intervals or human-evaluation findings. No price or hosted access option is described; the documented path is to download the materials and run the evaluation locally.

## What the archive adds
This is a new benchmark release in a recurring research-artifact thread, rather than a continuation of the same project. Earlier notes include [AfrIFact, a multilingual fact-checking benchmark](https://newruntime.com/posts/dataset-on-huggingface-code-on-github-accepted-at-5c6cec/) and [MVMAE, a radiology research-code release](https://newruntime.com/posts/code-at-github-com-agostini335-mvmae-accepted-to-1e2613/). Those projects address different tasks; their relevance is the pattern of papers accompanied by public artifacts, not shared methods or findings.

## Trend signal
No supplied SITE_PATTERNS thesis genuinely fits this release. MER-Bench provides evaluation code and artifacts for a specific multimodal task, but the source does not show a broader shift in agent interoperability, runtime security, verification capacity or another listed pattern. It is better treated as a bounded research resource than as evidence for a wider industry trend.

## Connections to other events
The project sits alongside other research releases that pair code or datasets with conference acceptance, including the archived AfrIFact and MVMAE notes. The connection is limited to the release format: MER-Bench concerns emotional transformation of memes, while those earlier projects address multilingual fact-checking and radiology representation learning. The repository itself names several image-generation systems in its example output directories, but the supplied material does not compare their results or establish any relationship between their developers and the benchmark authors.

## Why it matters
For teams building multimodal assistants or image-editing workflows, the benchmark makes a useful evaluation question more explicit: can a system alter emotional tone without changing what the meme depicts or how it communicates? Generic image-quality checks and sentiment labels may miss that trade-off. A multi-axis setup can help teams inspect whether an apparent improvement in tone came at the cost of semantic or stylistic fidelity.

The practical value depends on the benchmark’s coverage and the validity of its judgments, neither of which can be assessed from the supplied repository excerpt alone. Still, the published scripts and pre-generated outputs lower the friction of inspecting the task and attempting a reproduction. Teams should treat results from this benchmark as one task-specific signal, not as a general measure of multimodal capability or readiness for user-facing deployment.

## Limits and uncertainty
The supplied repository material does not state the number of examples, how the paired image-text data were collected, the exact scoring procedures, or whether judgments are automated, human, or mixed. It also provides no numerical results or model ranking in the excerpt. The presence of pre-generated outputs does not by itself establish that all models were run under comparable settings, and the listed reproduction commands do not show compute requirements or runtime. Acceptance to EMNLP 2026 is reported by the project README, but is not a substitute for examining the paper and its methods. The source supports describing an available benchmark and workflow, not claiming that it is representative, definitive or predictive of deployed performance.

## Skill to build now
When evaluating an image-editing model, write down separate acceptance criteria for the intended change and the content that must remain invariant. For a meme-reappraisal experiment, that could mean checking target emotion, scenario preservation and style preservation independently rather than collapsing them into one subjective “looks better” score. MER-Bench’s stated axes offer a concrete starting point, but teams should define their own rubric and validate it against their intended users before relying on scores.

## Reader takeaway
MER-Bench turns meme reappraisal into a reproducible multimodal evaluation task, with separate attention to emotional control, content preservation and quality. Its repository provides materials for inspection and reproduction, but the supplied source does not provide enough results or methodological detail to judge benchmark strength or model performance. Treat it as a useful research artifact to examine, not a settled scorecard.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Github](https://newruntime.com/topics/github.md): Explore the github topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)