Bidirectional Loop between:
Human Signals and Trustworthy LLMs

“I turn human signals into evaluation, alignment, and adaptation for LLM-powered AI systems”

Xin Sun
13 July 2026

Specially Appointed Assistant Professor · NII, Japan

Research Background

interdisciplinary background

Human-Centered AI

(Natural language processing, NLP &
Human-computer interaction, HCI)

Cognitive Science

(Mixed-methods empirical studies
with behavioral & physiological sensing)

Trustworthy Human–AI Interaction

from both sides — humans align AI ⇆ AI augments humans

ContextsHealthcareMental SupportDecision-Making Support

Background: Education & Experience

An interdisciplinary path — computer science, NLP & cognitive science, across four countries.

Appointments & Education

2025.05 – nowNational Institute of Informatics (NII)Assistant Professor · Tokyo, Japan

2026.03 – nowAmsterdam University Medical CenterTechnical Advisor (part-time) · Netherlands

2021.04 – 2025.05University of AmsterdamPh.D. · Netherlands

2017.11 – 2020.12Heidelberg UniversityM.Sc. in Computer Science (NLP) · Germany

2011.09 – 2015.07Xidian UniversityB.Sc. in Computational Mathematics · China

Research & Industry Experience

2023.08 – 2025.04CWI — Institute for Mathematics & Computer SciencePh.D. Researcher · Netherlands

2020.05 – 2020.10Alibaba GroupNLP Algorithm Engineer Intern · Hangzhou, China

2019.04 – 2019.10SAPData Scientist Intern · Walldorf, Germany

2018.07 – 2018.09Tsinghua University, Applied Research InstituteMachine Learning Researcher Intern · China

2015.08 – 2016.04BoschData Analyst · China

Research Motivation: Human Signals ⇄ LLM Support — Neither Is Always Right

LLMs are trained on human data to support humans, however:

Humans

(cognition & behavior)

LLMs

(behavior & mechanism)

evaluates both humans & LLMs

Human signals✗ not always ground truth

LLM support✗ not always appropriate

Trustworthy Human–AI Loop

3 / 11

Research Framework: One Loop, Three Layers — Mechanism · Method · Modeling

Turn human signals into a trustworthy Human ⇄ LLM loop.

Humans

LLMs

Objective: Learn the right things from humans · provide the right support back to humans

— with both directions trustworthy —

4 / 11

Evaluate

LLM as Judge

Diagnose how humans and LLMs judge — and where they fail alike.

5 / 11

1 · Evaluate: Can We Trust the Judge — Humans and LLMs?

LLMs learn from humans, so do they “think” like us? We probe both of them
in evaluation tasks via three steps:

Same heuristic reliance, same mechanism — but LLMs weigh evidence not fully as we humans do.

next · Align — teach LLMs to learn from humans selectively →

6 / 11

1 · Evaluate: LLM-as-a-Judge vs. Human Judge

DomainHealth Information

Research Question: when the same content is labeled ‘human’ vs ‘AI’-written, do humans and LLM judges shift their trust the same way?

Experimental Example

Label: Human-written

“Daily walking helps lower blood pressure.”

identical content — only the source label flips.

click to flip the label

Trust in the same content · now labeled HUMAN · on a 1–7 scale

Human

5.4

GPT‑5.4

5.7

Claude Opus

5.5

Llama‑3‑70B

5.8

Qwen3‑30B

5.1

GPT‑OSS‑120B

5.6

GPT‑OSS‑20B

4.9

flip the label on the left → every bar drops under ‘AI’ — identical content, less trusted just for the label.

Takeaways: Humans and every LLM judge lean on the same shortcut — identical content is trusted more just for looking ‘human-written’.

Sun et al. Label Effects: Shared Heuristic Reliance in Trust Assessment by Humans and LLM-as-a-Judge (ACL 2026) Core A*
[b] Sun et al. Understanding Trust Perception in AI-Generated Information with Behavioral and Physiological Sensing (IJHCS 2026) Top Tier JCR-1

1 · Evaluate: Are the Judge’s Signals Independent — or One Correlated Verdict?

Research Question: does an LLM judge each dimension on its own — or as one single overall impression?

1Why it matters if every dimension score comes from one decision, judging ‘many dimensions’ is meaningless — nothing is weighed dimension by dimension.

2How we test · grade this answer “Vitamin C cures a cold — just take a double dose.”✗ actually false

told it’s from

3 a person judges each dimension on its own — the LLM merges them into one

1Judging many dimensions

independent scores — or one call?

Human

Fluency

Accuracy

Safety

scores differ — accuracy drops

LLM

Fluency

Accuracy

Safety

all alike — one call

the LLM’s three scores collapse into one

2Trust vs Truth

kept apart, the way people do?

Human

Trust

Truth

keeps the two apart

LLM

Trust

Truth

locks the two together

humans separate them; the LLM fuses them

Takeaways: a person weighs each dimension on its own; the LLM rolls them into one correlated feeling.

Sun et al. When Trust Meets Truth: Trust–Truth Separability in LLM-as-Judge (EMNLP 2026) Core A*

1 · Evaluate: Beyond Behaviors - Mechanistic Evaluation for LLMs and Humans

what each judge reads label: “written by a human” “daily walking lowers blood pressure.” → both judge by the label, not the claim itself

Human · eye-tracking · where people look

Label cue

over-attended

Content

less attended

LLM · model internals · where attention flows

Label cue

over-weighted

Content

less weighted

Takeaway: human gaze and LLM attention show the similar mechanistic patterns — both over-attend the label cue, not the content.

[a] Sun et al. Label Effects (ACL 2026) Core A* — behavioral results, mechanistic analysis across LLM families

Align

LLM as Learner

LLMs selectively learn from human expertise — resist the bias that the Evaluate exposed.

7 / 11

2 · Align: Learn Human Expertise, Keep Out the Bias

Human data holds both expertise and bias — so we align LLMs selectively: keep the “good” signals, drop the bias.

Human data ✗ bias ✓ expertise

LLM inherits human bias

LLM learns from expert knowledge

Aligned LLM ⊘ bias — kept out ✓ expertise

selective alignment — only the “good” signals pass

a right model is still one-size-fits-all — next: how to adapt LLMs to each user's live state? →

8 / 11

2 · Align: Domain-Specific Benchmarks Creation

Use CasePsychotherapy · Mental Health

Research Question: what do expert strategies look like — the data an LLM should learn from and be tested on?

1Collect & code real sessions

real psychotherapy · EN + Dutch · every turn labeled by therapist experts

ClientI know I should cut down on drinking… but I just can’t.

TherapistYou feel stuck between wanting change and old habits.Reflection

ClientExactly. And I don’t know where to start.

TherapistWhat has worked for you, even a little, in the past?Open Question

2Organize into a strategy tree

expert codes + evidence-based scripts → phases & strategies

Domain-specific strategy and script Engaging Evoking Planning Reflection Open Question Affirmation ▲ predict next Elicit Change Talk Advice w/ Permission

Takeaways: makes human expertise measurable — defining what to align LLMs to.

[d] Sun et al. Eliciting Motivational Interviewing Skill Codes in Psychotherapy with LLMs: A Bilingual Dataset and Analytical Study (COLING 2024) Core B
[c] Sun et al. Script-Strategy Aligned Generation (ACM CSCW 2025) Core A

2 · Align: Supervise Models with Domain Expertise, Not Simply Imitate

Use CasePsychotherapy · Mental Health

Research Question: align LLM to domain-specific expertise and strategies, not only raw human imitation.

Domain Adaptationbase model · the domain benchmark

base LLM→domain benchmark⇒domain-aware model

PredictionCoT → one strategy

“I know I should exercise, but I’ve got no time.”

client voices resistance → affirm the effort ⇒ Affirmation

Generationconditioned response

“Even spotting that barrier is already a real first step.”Affirmation

why SSAG — the only one strong across the board, where the others trade off

Rule-based system

Adherence

Flexible

Controllable

Explainable

Effective

Pure LLM (no align)

Adherence

Flexible

Controllable

Explainable

Effective

Fine-tuning

Adherence

Flexible

Controllable

Explainable

Effective

SSAG (ours)

Adherence

Flexible

Controllable

Explainable

Effective

Sun et al. Script-Strategy Aligned Generation (ACM CSCW 2025) Core A
Sun et al. Rethinking the Alignment of Psychotherapy Dialogue Generation (COLING 2025) Core B

2 · Align: Multi-Agent Therapeutic Dialogue Generation

Research Question: expert data is scarce — can LLM agents generate it, with domain-specific strategies under control?

Coordinator Agent

coordinates both agents · sets each turn’s strategy

Therapist
Agent

Client
Agent

strategy · Open Question
“What makes exercise hard for you?”

coded · sustain talk
“Honestly, I just don’t have the time.”

Multi-agent simulation · coded turn by turn

Benchmark Results

6Kcoded dialogues1Kstory pairs12MI code strategies6LLMs benchmarked

Evaluated on

lexical quality
domain-strategy adherence
LLM-judge + expert review

Takeaways: with strategy control, LLM agents turn scarce expert data into clinically-plausible alignment supervision — at scale.

[j] Sun, et al. StoryMI: Steerable Multi-Agent Therapeutic Dialogue Generation. co-authored (ACL 2026)

2 · Align: How LLMs Learn the “Bad” Human Signals

DomainHealth Information

Research Question: how do LLMs learn the “bad” signals from humans? — through alignment training:

1Why it matters We train models on human preferences — but preferences carry bias. Which training step actually learns it?

2How we test it

plant · a fake cue always marks the “preferred” answer

✓ preferredcue: “human” ✗ rejectedno cue

→

train 3 ways Pre-train → SFT → DPO
same base model

→

measure does it now judge by the cue, not the content? → the cue gap (vs a decorrelated control)

Pre-train next-token prediction

bias installed · leastjust reads the text — exposure only

SFT supervised fine-tuning

bias installed · moderateimitates the answers

DPO preference optimization

bias installed · significantrewards the human-preferred choice — bias included

3Evidence installed bias Δ (gap − base) — flat until DPO, then it spikes

Takeaways: installed by preference, not by exposure — Pre-train · least › SFT · moderate › DPO · significant.

[j] Sun et al. Installed by Preference, Not by Exposure: Heuristic Cue Biases in LLM Judges (AAAI 2026, under review) Core A*

Adapt

LLM as Partner

Adapt to each user's live state — support the person, not the average.

9 / 11

3 · Adapt: Why Adapt — From an Aligned Model to the Right Support

Evaluate ✓ + Align ✓ make the model right — but a “good” model is still one-size-fits-all

but the same user’s state shifts turn to turn —

the user responds — and the loop runs again, every turn

support tuned to this user, at this moment — not the average user

next · Vision — the full trustworthy loop →

10 / 11

3 · Adapt: Sense Human Signals in Real Time

Use CaseHealth Decision Support

1Sense

Research Question: How can multimodal signals reveal a user’s state during real-time interaction?

Gaze — the eye-tracker

fixation countfixation durationdwell per region (AOI)scanpath

where attention goes, moment to moment

Physiology — ECG · EDA · skin temp

heart rateHRVskin conductanceskin temperature

arousal & effort — signals people cannot mask

Behavior — interaction logs

cursor & clickstyping pacedaily steps

overt actions in the task

behavior shows what people do · physiology reveals what they cannot mask — continuously, not only after the task.

[b] Sun et al. Understanding Trust Perception in AI-Generated Information with Behavioral and Physiological Sensing (IJHCS 2026) Top Tier JCR-1

3 · Adapt: Predict Human States from Multimodal Signals

Use CaseHealth Decision Support

2Predict

Research Question: can the fused signals predict a user’s hidden states — load, trust, confidence — in real time?

ML models trained per state · validated against self-reports

Multimodal signals

Gaze behaviorsfixations · saccades · dwell

Pupilpupil diameter

ECGBPM · HRV (RMSSD)

EDAskin conductance

User characteristics

traitsdemographicsAI literacy

prediction fluctuates ~robust ✓

Cognitive states · model output

Cognitive loadhigh · noisyhigh · stable

Trustmoderate · noisymoderate · stable

Decision confidencemedium · noisymedium · stable

Affective statesArousal→ correlated with physiology

BehaviorsSource perception→ gaze ↔ trust: highly correlated

Takeaways: signals alone fluctuate — fused with user characteristics, states become readable & stable — the control signal for adaptation.

[b] Sun et al. Understanding Trust Perception in AI-Generated Information with Behavioral and Physiological Sensing (IJHCS 2026) Top Tier JCR-1
Sun et al. Eyes Can’t Always Tell (ACM UMAP 2026) Core A

3 · Adapt: Adaptive Support that Augments Humans

3Adapt

Adaptive support fits each individual, not the average — multimodal sensing × domain-tuned LLMs × adaptive interfaces close the loop.

AI adapts shown, and scaled to your load so you trust it — not overwhelmed

AI adapts disclosed confidence says its uncertainty out loud so you trust it the right amount

AI adapts timing & nudges support at the right moment so you act in time

AI adapts text · voice · robot · VR so you engage the way that fits

Sense

Predict

Adapt

Person wearing a VR headset in an interactive virtual environment

Adaptive LLM System

Body sensingGaze · Physiological signals · HR · ECG · Pupil

Multimodal signals: speech + vision

Movement · activity data

Fusion + Prediction Multimodal signal processing → affective & cognitive state prediction

User-conditioned LLM dialogue in specific domains with safeguards

Adaptive Generation & Interface multimodal sensing × domain-tuned LLM × adaptive support

[f] Sun et al. Interface Matters: Exploring Human Trust in Health Information from LLMs via Text, Speech, and Embodiment (ACM CSCW 2025) Core A
[h] Sun et al. From Search Agents to Dissemination Interfaces: Understanding Human Trust in Health Information from Conversational Search (BIT 2026) JCR-1
[k] Sun et al. When LLM Rationales Become User-Facing: Effects on Trust Perception, Decision-Making, and Gaze Behaviors (IJHCS 2026, under review) Top Tier JCR-1

Vision: Toward Human-Grounded, Reliable and Trustworthy LLM-powered AI Systems

from humans to LLMs · learn the right things

Humans

signals — behavior · cognition · physiology

LLMs

reliable judge · aligned learner · adaptive partner

from LLMs back to humans · provide the right support

learn the right things from humans · give the right support back

11 / 11

Thank you so much
for your time and patience!

Bidirectional Loop between:Human Signals and Trustworthy LLMs

Research Background

Human-Centered AI

Cognitive Science

Research Motivation: Human Signals ⇄ LLM Support — Neither Is Always Right

Humans

LLMs

Research Framework: One Loop, Three Layers — Mechanism · Method · Modeling

Evaluate

1 · Evaluate: Can We Trust the Judge — Humans and LLMs?

Align

2 · Align: Learn Human Expertise, Keep Out the Bias

Tracing

Benchmark

Approach

Scaling

Adapt

3 · Adapt: Why Adapt — From an Aligned Model to the Right Support

Sense

Predict

Adapt

Vision: Toward Human-Grounded, Reliable and Trustworthy LLM-powered AI Systems

Humans

LLMs

Evaluate → Signal Diagnosis

Align → Selective Alignment

Adapt → Closed-Loop Augmentation

Bidirectional Loop between:
Human Signals and Trustworthy LLMs