# Coding rules Every rule here was written when the ambiguity was first hit, not reconstructed afterwards. This file is published as an appendix so any row can be re-derived. --- ## Unit of analysis The unit is the **publication programme** — a publisher plus a recurring title — not the publisher and not the individual edition. Reason: cadence, format, gating and method disclosure are properties of a programme, not of an organisation. Deloitte Greece runs three distinct programmes (Hospitality Reimagined, Future of the Greek Hotel, Greek CFO Survey) with different formats and audiences; collapsing them to one row would lose exactly the variation the paper is about. Both counts are reported: **70 programmes across 65 distinct publishers** (as of 5 Aug 2026). --- ## Inclusion All three must hold: 1. **Greek or substantively Greek-operating.** Includes Greek-founded companies publishing in English from abroad (Workable, Epignosis, Obrela, Petrofin) and global firms' Greek arms publishing Greek-specific research (Deloitte Greece, Randstad Greece, NielsenIQ Greece). Excludes a global report with a Greek press release but no Greek data. 2. **Original research.** Own data, own survey, or a study the publisher commissioned. Excludes translated global editions, news roundups and curated link lists. 3. **Free in whole or as a substantive teaser.** A teaser must carry a finding, not just a table of contents. ## Exclusion decisions taken | Case | Decision | Reason | |---|---|---| | State bodies (ELSTAT, EETT, Bank of Greece, EEEP) | **Excluded** | Statutory publication is not the behaviour being studied | | Regulator research quoted by companies (EEEP gambling data) | Excluded | Same | | Universities publishing alone | Excluded | Included only as `co_signer` on a company- or association-funded programme | | Business associations (SEV, GSEVEE, GRECA, SEAA, INSETE, EAEE, SEVT, AmCham) | **Included** | Company-funded bodies publishing on members' behalf — the same behaviour with the cost pooled. Flagged by `co_signer=association` or by publisher name | | diaNEOsis | Included | Privately funded by businesses, publishes free by constitution | | Free software tools with dashboards (Cardlink apollo, Nexi eMerchants) | Excluded | A tool is not a publication | | Companies whose figures reach the press only via filed accounts (efood) | Excluded | Journalism about a company is not publication by it | | MRB, Kapa Research | **Held, not coded** | Confirmed as research houses; no free recurring publication programme verified yet. Revisit at stage 4 | | Colliers Greece, Prosperty | Held, not coded | Research page not confirmed | --- ## Field rules **`format`** — six codes after the round-1 recode: | Code | Definition | |---|---| | A | **Own operational data** — the publisher's own transactions, listings, logs, portfolio or fleet | | B | **Deep practical whitepaper** — a long document teaching the reader to do their job better | | C | **Commissioned survey** — new primary collection from respondents | | D | **Institute-authored footprint study** — economic contribution modelling where an **independent institute is the author**. Author independence is the defining feature; a self-authored footprint claim is A or F with a policy motive | | E | **Free teaser over paid data** — findings free, dataset sold | | F | **Desk / analytical study** — original analysis of third-party data with no new collection | Where a programme could be two, the **data basis decides**: own operational data → A; new collection → C; neither → F. If the full dataset is sold, E overrides everything else. **`editions_to_date`** — count of published editions. `editions_evidence` records whether the count is `evidenced` (stated by the publisher or countable from the archive) or `estimated` (inferred from cadence and first-seen year). **Estimated counts are not used in any headline claim** — only in the cadence distribution, and the split is reported. **`access`** — `ungated` (no form), `ungated-teaser` (findings free, full data sold, no form), `email-gated` (form for the full document), `teaser-plus-paid` (free summary, priced full report). **`n_disclosed`** — `yes` only if a sample size or corpus size appears in the public artefact. `partial` if a scale claim exists without a number ("thousands of listings"). **`method_disclosed`** — `yes` requires at least: what the data is, the period, and how it was collected. Naming the data source alone is `partial`. **`co_signer`** — the strongest external name on the artefact: `institute` > `university` > `association` > `consultancy` > `none`. **`motive_primary`** — inferred, never asserted, from four observable signals (see below). One value only; the runner-up goes in a note. **`csr_framed`** — `yes` if the artefact itself frames the work as contribution, footprint, sustainability or responsibility. This is coded **separately from motive** so the CSR question can be answered two ways: how many are *framed* as CSR, and how many are *motivated* by licence-to-operate. --- ## The motive decision rule Four observable signals, applied in order. The rule is published in the paper. | Signal | Reading | |---|---| | **Author** | Independent institute or university on the cover → licence-to-operate or policy. Self-authored → commercial | | **Gate** | Email-gated → demand-gen, always. Ungated → any other motive | | **Headline metric** | Jobs, GDP share, multipliers, contribution → licence-to-operate. Behaviour, prices, benchmarks, indices → category-ownership. Practical how-to → demand-gen | | **Addressee** | Government, media, society → policy. Buyers → demand-gen or category-ownership | Ties resolved by addressee. `pr-amplification` is used where the artefact is a press release rather than a document and no deeper purpose is visible. `teaser-to-paid` overrides where the research exists to sell the dataset. **Known weakness:** this infers intent from artefacts. A publisher may hold a motive the artefact does not reveal. Stated as limitation #2; `csr_framed` is coded independently so a reader can re-derive the CSR answer without trusting the motive column. --- ## Reliability check — run 5 Aug 2026, GATE FAILED Systematic sample, every 5th programme from G001 (n=14, 20%). Re-coded from the same source artefacts without reference to the original codes. Full record in `reliability-check.csv`. **This is intra-rater (test–retest) reliability by a single coder, not inter-rater.** It catches rule-application inconsistency, which is what it found. It cannot catch a rule that is wrong in a way one coder would repeat. Inter-rater remains open and is stated as limitation #6. **Row-level agreement: 9/14 = 0.64.** The workflow gate is 0.80, so the schema is tightened and the corpus recoded before any analysis is published. Field-level agreement was much higher — 5 changed decisions out of ~70 — which locates the problem precisely: not sloppy coding, but three under-specified rules. ### Fix 1 — the schema has no code for desk analysis (`schema-gap`, 1 case, ~6 rows affected) Bank and institute sector studies (NBG, Alpha, Eurobank, Piraeus Holdings, KPMG sector surveys, IENE) are neither commissioned surveys nor own operational data. They were all coded `C` by default, which is wrong and inflates the survey count — the corpus's single most quoted figure. **New code `F` = desk/analytical study**: original analysis of third-party data, no new collection. Recode candidates: G034, G035, G036, G038, G043, G062, and a full sweep of every `C` row for the same error. ### Fix 2 — motive for association publishers (`motive-rule`, 2 cases) Where the publisher is an association or confederation and the addressee is the state, the code is `policy`, not `category-ownership`, even when the artefact also serves members commercially. Applies to G006 and G011; sweep all association rows. ### Fix 3 — two inclusion rules applied inconsistently (`inclusion-rule`, 2 cases) - **G041 EY-Parthenon CEO Outlook** — a global survey with Greek framing. Inclusion rule 2 excludes these unless a Greek data cut is published. Check, then keep or drop. - **G056 Skywalker.gr** — a salary calculator and listings data. The "a tool is not a publication" rule already excluded Cardlink apollo and Nexi eMerchants; applying it consistently means dropping this row. Both were errors of consistency, not judgement, which is exactly what a reliability check exists to surface. --- ## Reliability round 2 — run 6 Aug 2026, GATE PASSED Fixes 1–3 applied to the whole corpus, then a **fresh** systematic sample drawn with a different start (every 5th from G003, n=14) so the check was not simply re-testing what had just been repaired. Record in `reliability-check-2.csv`. **Row-level agreement: 13/14 = 0.93**, against the 0.80 gate. One disagreement, and it produced a fourth rule: **G003 Hosthub's GDP-share research was coded D**, but D is an institute-authored format and this is self-authored advocacy from the publisher's own data. Recoded A with a policy motive, and the definition of D was tightened to make author independence explicit. No other row was affected. Two rows in the round-2 sample (G038, G058) were spot-checks confirming fix 1 had landed in the corpus, not independent tests. **Corpus after recode: 68 included, 2 excluded, 64 publishers.** The most consequential change: commissioned surveys fell from a reported 36 of 70 to **26 of 68**, because eleven desk analyses had been defaulting into C. That figure was going to be the paper's headline. --- ## Reliability round 3 — run 6 Aug 2026, INTER-RATER, GATE FAILED ON ONE COLUMN The first genuinely independent coding. The whole corpus — all 68 rows, not a sample — was coded by a second agent (DeepSeek `deepseek-chat`, temperature 0, single pass) working from the rules in this file verbatim plus nine observable fields per programme, blind to the original codes. Record in `inter-rater-check.csv`, summary in `inter-rater-summary.json`. | Column | Raw agreement | Cohen's κ | Verdict | |---|---|---|---| | `format` | 85.3% | **0.80** | Meets the 0.80 gate | | `csr_framed` | 89.7% | 0.34 | High agreement, κ unstable — skewed marginal | | `motive_primary` | 51.5% | **0.34** | **Fails** | ### The three causes, diagnosed rather than assumed **1. This file was incomplete.** The corpus uses eight motive codes; this file defined six. `csr-marketing` (4 rows) and `employer-brand` (1 row) appeared in the results table and in no rule, so those five rows could not be matched by any second coder. `csr_framed` likewise carries a third value, `partial` (2 rows), against a rule that described it as binary. Both are now documented below. Excluding the five affected rows moves κ only to 0.376 — a real defect, not the main cause. **2. The gate signal was read as a biconditional.** Eleven of the 34 motive disagreements are identical in shape: coder 1 `demand-gen`, coder 2 `category-ownership`, and **all eleven are `access=ungated`**. The rule states email-gated → demand-gen. It does not state the converse, and it does not say what ungated implies, so the decision falls to signals 3 and 4 — where a consultancy's free sector report is honestly both a "practical how-to" and a "benchmark". **The rule under-determines a whole class of publisher.** **3. The column may not be reliably codeable from artefacts.** Motive is inferred. κ 0.34 is the measurement of what that inference costs. ### Disposition - The corpus was **not recoded**. Recoding after seeing the disagreements converts a reliability check into a tuning exercise. - Every motive-derived figure in the paper is now labelled a single-rater judgement at κ 0.34, indicative rather than measured. - The CSR answer is unaffected: `csr_framed` was coded independently of motive from the start for exactly this reason, and holds 89.7% raw agreement. Its low κ is the well-known paradox of a skewed marginal — 59 of 68 rows share one value, so chance agreement is already 84.5%. Raw agreement is the more informative statistic here and both are reported. - **The second coder was a language model, not a trained human researcher.** This tests whether the published rule reproduces a decision, which is what a published rule is for. It is not equivalent to human inter-rater coding and is reported as the weaker instrument it is. ### The full motive code list, previously undocumented | Code | Definition | |---|---| | `category-ownership` | Owning the recurring number in a category | | `demand-gen` | Generating or qualifying commercial demand | | `policy` | Addressed to the state or to public debate | | `teaser-to-paid` | Free findings existing to sell the dataset | | `pr-amplification` | A press release rather than a document; no deeper purpose visible | | `licence-to-operate` | Defending the publisher's right to operate | | `csr-marketing` | Contribution framing used as brand marketing rather than to defend a licence | | `employer-brand` | Addressed to candidates rather than buyers or the state | `csr_framed` takes `yes`, `no`, or `partial` — the last where contribution framing appears in part of the document only. --- ## Open items - Arrange genuine inter-rater coding on a 20% sample. Both rounds so far are intra-rater. - Arrange genuine inter-rater coding on a 20% sample. - Convert `estimated` edition counts to `evidenced` where an archive can be counted. - Retry the two likeliest false-negative searches: industrial footprint studies (Q019) and Greek IT/cyber vendors (Q028). - Resolve the held cases: MRB, Kapa Research, Colliers Greece, Prosperty, AmCham employer-skills survey.