By name: psych-experiment description: Psychology experiment design and data collection planning. Use when designing questionnaire studies, laboratory experiments, or online surveys; selecting validated Chinese scales; planning participant recruitment; preparing IRB/ethics applications; or designing attention checks and data quality controls. user-invocable: true
Psychology Experiment Design
This skill guides the design of psychology experiments and data collection procedures.
Study Types and Design Considerations
Survey / Questionnaire Study
Best for: attitudes, traits, self-reported behaviors, retrospective data
Key design decisions:
- Cross-sectional vs. longitudinal (time points?)
- Between-subjects vs. within-subjects (same participants fill all scales?)
- Order of scales (counterbalance if within-subjects to avoid carry-over)
Laboratory Experiment
Best for: causal inference, reaction time, physiological measures, behavior observation
Key elements:
- Manipulation check (verify the IV was successfully manipulated)
- Randomization (use random.org or R
sample()for condition assignment) - Blinding (are experimenters blind to condition?)
- Standardized instructions and debrief script
Online Experiment (Qualtrics)
Best for: large samples quickly, remote participants
Qualtrics-specific considerations:
- Use "Page break" between blocks to avoid scrolling issues
- Use "Timing" element to detect speeding
- Use display logic for skip patterns
- Preview on mobile (many Chinese participants use phones)
Validated Chinese Scales Database
Personality / Individual Differences
| Scale | Chinese Version | Source | Items |
|---|---|---|---|
| Big Five (NEO-PI-R) | 中文修订版 | Costa & McCrae; Wang et al. (2004) | 240 items |
| Big Five (BFI-44) | 中文版 | John et al.; Schmitt et al. (2007) | 44 items |
| Conscientiousness (BFAS) | — | DeYoung et al. (2007) | 10 items/facet |
| Self-control (Brief) | 中文版 | Tangney et al.; Ent et al. | 13 items |
Emotion / Well-being
| Scale | Chinese Version | Source | Items |
|---|---|---|---|
| PANAS | 中文版 | Watson et al.; Huang et al. (2003) | 20 items |
| DASS-21 | 中文版 | Lovibond & Lovibond; Antony et al. | 21 items |
| SWLS (Life Satisfaction) | 中文版 | Diener et al.; Luo et al. | 5 items |
| PHQ-9 (Depression) | 中文版 (Patient Health Q.) | Spitzer et al.; He et al. | 9 items |
Social Psychology
| Scale | Chinese Version | Source | Items |
|---|---|---|---|
| SES (Self-esteem, Rosenberg) | 中文版 | Rosenberg; Wang et al. | 10 items |
| Attributional Style Q. | — | Peterson et al. | varies |
| Need for Cognition (NFC) | 中文版 | Cacioppo & Petty | 18 items |
Note: Always verify the Chinese validation study before using. Search via 工部 using:python3 lit_search.py search "Chinese validation [scale name]" --source pm
IRB Checklist (Ethics Application)
When preparing an IRB/ethics application, ensure all items are addressed:
# IRB Pre-submission Checklist
## Study Information
- [ ] Study title and PI information
- [ ] Study duration and timeline
- [ ] Funding source (if any)
## Participants
- [ ] Target population and inclusion/exclusion criteria
- [ ] Recruitment method (poster/online/subject pool)
- [ ] Compensation details (amount, form, timing)
- [ ] Estimated number of participants
- [ ] Vulnerable populations? (children, patients, students of PI — needs extra justification)
## Procedures
- [ ] Complete description of all procedures
- [ ] Duration of each session
- [ ] Any deception used? → Requires justification + debrief plan
- [ ] Any physical procedures (blood draw, physiological measures)?
## Risks and Benefits
- [ ] Potential psychological distress? (e.g., trauma-related content)
- [ ] Privacy risks?
- [ ] Direct benefits to participants?
- [ ] Benefits to society/science?
## Data Privacy
- [ ] Data de-identification plan
- [ ] Data storage location and duration (typically 5-10 years)
- [ ] Who has access to identifiable data?
- [ ] Data sharing plan (OSF? upon request?)
## Informed Consent
- [ ] Consent form includes: study purpose, procedures, risks, benefits, confidentiality, voluntariness, withdrawal rights, contact info
- [ ] Consent is obtained before ANY data collection
- [ ] Online study: digital consent (clicking "agree") acceptable at most institutions
Participant Recruitment Options (China)
University Subject Pool
- Best for: student samples, convenience sampling
- Typical compensation: course credits or ¥20-50
- Timeline: 2-4 weeks for quota if pool is active
- Limitation: WEIRD sample, low generalizability
Prolific (International)
- Best for: diverse adult samples, English-language studies
- Cost: ~£8-12 per hour of participant time (standard rate)
- Timeline: days to weeks depending on quota
- Requires: Prolific account, study in English
WeChat / Social Media Recruitment
- Best for: quick convenience samples, Chinese-speaking participants
- Cost: typically ¥10-30 in red envelope (红包) compensation
- Timeline: 1-2 weeks if network is large
- Risk: non-random, potential volunteer bias
Commercial Panels (问卷星, 京东众包, etc.)
- Best for: large quota quickly
- Cost: ¥15-50 per response depending on length
- Timeline: days
- Risk: data quality concerns; need strong attention checks
Attention Checks and Data Quality
Types of Attention Checks
- Instructed response items: "Please select 'Strongly agree' for this item"
- Logical consistency check: Reverse-score the item and check consistency
- Open-ended speeder check: Minimum character count response
- Timing check: Flag participants completing <50% of expected time
Exclusion Criteria Template
Participants will be excluded from analysis if they:
1. Complete the survey in less than [X] minutes (< 50% of pilot median)
2. Fail [≥ 2 / ≥ 1] attention check item(s)
3. Provide non-sensical responses to open-ended items
4. Have [X]% or more missing data on key measures
Qualtrics Setup Tips
- Set "Force response" only for critical items (prevents accidental progress)
- Use "Prevent ballot stuffing" (IP-based duplicate prevention)
- Add Progress Bar (improves completion rate)
- Mobile-friendly: avoid matrix questions with >5 columns on mobile
- Consider using "Embedded data" to track UTM source for different recruitment channels
Sample Size Reporting Format
Sample size was determined a priori using G*Power 3.1 (Faul et al., 2009).
For [test type], assuming an effect size of [d/f/r] = [value] (based on [citation / meta-analytic estimate]),
α = .05 (two-tailed), and power = .80, a minimum of N = [number] participants per group
([number] total) was required. We targeted N = [number + buffer] to account for
anticipated data exclusions.