Methodology · review-evidence-v1.0.0

How Law Leaderboard ranks firm profiles by public review evidence

Law Leaderboard compares public firm profiles using rating, review depth, recommendation language, communication mentions, owner responses, consistency, and negative-sentiment signals. The result is a public-review evidence score, not a judgment about legal ability or likely outcomes.

A rank means “stronger under this disclosed public-review evidence formula.” It does not mean best lawyer, best legal skill, best results, or best fit for a particular case.

Current scope

The current dataset covers 1,130 identity-matched public Google Business firm profiles across 25 ranking markets. It tracks 527,168 public reviews, contains 450,536 retrieved review records, and analyzes 365,756 review texts. 1,100 profiles currently pass the scoring gate; the others remain visible with no score or rank.

Active consumer release llb-consumer-intelligence-2026-08-12-r4 publishes 121 practice/city comparisons and 1,029 distinct qualified offices. Each assessed cell targets 50 discovered office profiles, and a page is published only with at least eight qualified offices.

1,130
Profiles covered
25
Ranking markets
527,168
Reviews tracked
365,756
Texts analyzed
1,100
Profiles eligible

Entity graph · entity-graph-v1.0.0

How brands and offices are resolved

Release llb-intelligence-2026-08-09-r2 distinguishes a law-firm brand from its offices: 464 brands, including 28 multi-office brands, and 523 offices are represented. Stable opaque Law Leaderboard brand, office, and firm IDs support traceability across versioned releases, while raw provider identifiers remain private.

464
Normalized brands
523
Verified offices
23
Exact candidate links
24
Ambiguities quarantined
Conservative match rule: only an exact Google Place ID or CID can automatically link a candidate observation to an existing office. Domain, name, address, phone, brand relationships, and proximity can support investigation, but none can silently merge offices. The current candidate pool contains 568 observations; ambiguous cases remain in quarantine until resolved.

Review intelligence · review-themes-v1.0.1

What review themes are published—and withheld

The current classifier processed 229,403 usable texts from 275,801 review records and assigned 36,263 private theme labels across 11 themes. Only 3 themes pass every publication gate: Communication, Responsiveness and delays, Outcome language (unverified). Outcome language records what a reviewer claimed; Law Leaderboard does not verify a result, recovery, settlement, or verdict.

≥ 50
Analyzed texts per office
≥ 3
Theme matches per office
≥ 0.80
Average label confidence
≥ 0.80
Independent QA precision
≥ 10
Reviewed predicted positives
Theme QA positives reviewed QA precision Release status
Communication 55 96.4% Published
Responsiveness and delays 38 94.7% Published
Staff accessibility 27 44.4% Suppressed
Case handoffs 24 62.5% Suppressed
Intake experience 27 14.8% Suppressed
Fees 24 25.0% Suppressed
Process clarity 27 29.6% Suppressed
Follow-up 31 54.8% Suppressed
Timeliness 26 42.3% Suppressed
Compassion 39 79.5% Suppressed
Outcome language (unverified) 44 81.8% Published

Independent validation used gpt-4o-mini-2024-07-18 on a deterministic stratified sample of 483 reviews. Manual source inspection covered 27 positive-evidence records: 15 API agreements and all 12 API-disagreed positives for the published themes. Explicit evidence appeared in all 27 inspected records. Neither check establishes perfect labels or complete recall. 8 themes remain available for private classifier improvement but are absent from public office metrics in this release.

Versioned lineage and downloads

Every public metric is tied to release 2026.08.09-r2, source and corpus checksums, classifier and entity-graph versions, and an independent QA run. The manifest records byte counts and SHA-256 checksums for each file.

Manifest Data dictionary Office CSV Brand CSV Classifier QA Checksums

Trend suppression

Trend fields are prepared but suppressed unless at least two distinct, complete profile observations exist. A suppressed trend means the evidence window is incomplete; it does not mean review activity or Maps visibility was unchanged. The graph currently preserves 363,504 review-to-snapshot links for future comparable releases.

From discovery to ranking

1

Discover candidate firm profiles

The current consumer release targeted 50 discovered office profiles in each of 150 assessed practice-area-by-city cells across six practices and 25 cities. Stable provider identities and entity matching prevent one office from becoming multiple public profiles.

  • Maps discovery order identifies candidates; it does not rank review evidence.
  • Later Maps visibility snapshots remain a separate time-specific field.
  • Maps position never enters the score or its tie-breaks.
2

Collect source records

We collect observed public Google Business Profile fields and retrieve public review records through a third-party data provider. Tracked public review count, retrieved review records, and analyzed review text are three different coverage measures.

  • Tracked count is the public aggregate shown for the profile at collection time.
  • Retrieved count is the number of review records captured for analysis.
  • Analyzed count includes retrieved records with usable review text.
3

Derive aggregate review signals

Versioned deterministic rules identify explicit review language across 11 defined themes. The private graph retains all labels for QA and improvement; public office metrics are emitted only for themes that pass every sample, confidence, and independent-validation gate.

  • Independent OpenAI labels and a manual positive-evidence sample test the deterministic rules.
  • Classifier outputs may be incomplete or wrong, and a detected label never verifies the underlying claim.
  • Public files expose gated aggregates only; raw review text and reviewer identity remain private.
4

Apply the review and practice gates

A profile receives a score only when every eligibility field is present and at least 50 review texts were analyzed. Practice classification for a city membership also requires an explicit public-profile practice signal or sufficient context-classified review evidence. A market page requires at least eight offices that pass both gates.

  • Non-empty stable Place ID and a valid public rating from 1.0 to 5.0.
  • Tracked public review count greater than zero and at least 50 analyzed text reviews.
  • Recommendation, communication, owner-response, consistency, and negative-sentiment inputs all present.
5

Shrink inputs and calculate the score

Eligible profiles receive a weighted sum of seven 0-to-100 components. Rate-like signals are adjusted toward fixed dataset priors so smaller evidence samples have less influence. Component and weighted values are retained to three decimals before the final score is shown to one decimal.

  • The theoretical score range is 0.0 to 100.0.
  • The current eligible-profile range is shown below as a descriptive snapshot, not a permanent bound.
  • Every exported firm record includes the version, eligibility, component values, weights, and weighted points.

Review Evidence Score formula

Version review-evidence-v1.0.0 returns a theoretical 0.0-to-100.0 weighted score, displayed to one decimal. Component scores and weighted points are retained to three decimals before final rounding. In the current export, eligible scores range from 31.5 to 93.7, with a median of 72.20. Those observed values will change when the evidence changes.

Component Weight Reproducible transform Evidence meaning
Public rating 25% clamp((shrunk rating − 4.0) × 100, 0, 100) Uses tracked review count as the shrinkage sample size.
Review depth 10% clamp(100 × ln(1 + tracked reviews) ÷ ln(1 + 2,500), 0, 100) Rewards more observed public-review evidence with diminishing returns and a 2,500-review cap.
Explicit recommendation 20% shrunk recommendation percentage Measures explicit recommendation language in the analyzed text sample.
Communication mentions 15% clamp(shrunk communication percentage ÷ 40 × 100, 0, 100) Maps communication mentions onto a 0-to-100 component scale.
Public owner response 10% shrunk owner-response percentage Measures public replies to retrieved reviews, not communication during a case.
Consistency 10% shrunk rating-consistency index Uses the exported polarization/consistency measure.
Low concern 10% clamp(100 − 5 × shrunk negative-sentiment percentage, 0, 100) A lower adjusted negative-sentiment share produces a higher component score.

Fixed-prior shrinkage

Rate-like inputs use a fixed prior strength of 50: (raw × n + prior × 50) ÷ (n + 50) For rating, n is tracked review count. For recommendation, communication, owner response, consistency, and negative sentiment, n is analyzed text-review count. Owner response is converted from a 0-to-1 value into a percentage first.

Fixed priors: rating 4.875/5; recommendation 60.0%; communication 17.9%; owner response 67.6%; consistency 92.6/100; negative sentiment 3.0%.

Deterministic tie order

  1. Displayed score, descending.
  2. Analyzed text-review count, descending.
  3. Tracked public-review count, descending.
  4. Normalized firm name, ascending.
  5. Stable firm ID, ascending.

Maps rank and visibility do not enter the tie-break sequence.

Evidence-depth labels

Evidence depth describes the number of analyzed text reviews: insufficient below 50, minimum from 50–99, moderate from 100–249, strong from 250–999, and extensive at 1,000 or more. Only the 50-review eligibility floor affects whether a score exists; the label is otherwise descriptive.

Public-data privacy boundary

Public intelligence files contain no reviewer identity, review text, owner-response text, provider payload, raw Place ID or CID, street address, or phone number. They use opaque Law Leaderboard IDs for versioned traceability. Because raw provider identifiers remain private, the public files cannot independently reproduce the private identity-matching process. The versioned release is rights reserved and is not offered under an open-data license.

Known limitations

The score measures public-review evidence only. It does not measure legal ability, credentials, licensing, discipline, fees, results, settlement quality, specialty, case fit, or future outcomes.
Public Google ratings and review counts are third-party aggregates. Law Leaderboard does not author or independently verify them.
Retrieved review coverage can be lower than the tracked public count, especially on very large profiles.
Pattern and classifier outputs can miss context, sarcasm, ambiguity, or unusual phrasing and do not verify allegations or results.
Only 3 of 11 review themes pass the current publication gates. Suppressed themes may exist in the corpus but do not appear in public office metrics.
Independent API labels are a QA reference, not perfect ground truth. The sampled precision gate does not establish complete recall.
Google Maps discovery and visibility are query-, location-, device-, and time-specific. Neither is a quality measure.
The current 2026.08.12-r4 consumer release publishes 1,029 distinct qualified offices and 2,573 practice/city memberships. It withholds 29 assessed cells below the eight-office page gate.
Law Leaderboard does not accept paid placement.

Sources, Freshness & Limitations

Sources

  • • Observed public Google Business Profile fields and public review records collected through a third-party data provider.
  • • Aggregate review-analysis fields generated under analysis methodology version 2026.04.07.
  • • Review Evidence Ranking version review-evidence-v1.0.0, applied deterministically to the exported profile fields.
  • • Entity graph entity-graph-v1.0.0 and office benchmarks office-benchmarks-v1.0.0.
  • • Review-theme classifier review-themes-v1.0.1, independently checked with gpt-4o-mini-2024-07-18.

Freshness

  • • Business-profile observations include the 500-profile April 2026 base and a separate 23-office August 2026 directory extension.
  • • Review collection is profile-specific: 738 profiles were refreshed through August 2026; 392 profiles retain April 2026 review snapshots.
  • • Derived analysis uses each profile’s most recently captured review records; the directory extension was analyzed in August 2026.
  • • The versioned 500-profile public release was published August 3, 2026 and remains separate from the directory extension.
  • • The normalized intelligence release 2026.08.09-r2 was published 2026-08-09; its checksums and version identifiers apply only to that release.
  • • The depth-50 consumer release 2026.08.12-r4 was published 2026-08-12; its source cohort, market shards, coverage ledger, and checksums are versioned separately.

Limitations

Inspect the versioned aggregate release, field definitions, checksums, and usage terms before reusing a figure.

Open the dataset Explore rankings