BlueCollarBench

Trades/Roofing

RoofingTask RO-037masterCatchUS

Sub is torching modified bitumen cap sheet on a 400 sq ft flat porch roof.

Prompt as given to every model

Sub is torching modified bitumen cap sheet on a 400 sq ft flat porch roof. He's torching straight onto the new 5/8 plywood deck, torching right up to the stucco wall and around the old wood-framed skylight curb, and when he's done at 4 pm the crew packs up and leaves. What's wrong here?

Model answers

Pick up to two to compare side by side. Score chips fill in as the judge grades.

Nemotron 3 Ultra 550BScore —

Multiple critical failures:

  1. Direct-to-wood torch application (Fire Hazard & Code Violation): Torching directly to plywood violates IBC 1507.11.2 and manufacturer specs. You need a thermal barrier (e.g., 1/2" gypsum, DensDeck, or a nailed base sheet) between flame and combustible deck. This is a major fire risk—hot mopping or torching wood decks causes structure fires regularly.
  1. No base sheet: Modified cap sheet requires a mechanically fastened or adhered base sheet (per IBC 1507.11.2 & ARMA/NRCA guidelines). Cap sheet alone lacks tensile strength and puncture resistance.
  1. Wall flashing sequence wrong: Torching "up to stucco" implies no counterflashing. Correct: Run base/cap sheet up wall min 8" (IBC 1503.2), install termination bar/sealant, then stucco contractor cuts reglet and installs counterflashing. You cannot seal to raw stucco reliably.
  1. Skylight curb detail missing: Torching around a wood curb without a pre-flashed curb wrap, cant strip, and counterflashing guarantees leaks. Wood curbs require metal flashing integration per IBC 1503.2 / NRCA details.
  1. Cold-night cure abandonment: Modified bitumen needs time to cool/set without traffic or thermal shock. Leaving at 4 PM with no QA inspection (seam probes, adhesion checks) is negligent.

Stop work. Require: Fire watch during torch ops, approved thermal barrier/base sheet, proper wall/curb flashing details, and daytime completion with seam verification.

512 tokens · 6.9s

Judge rubric · pending

Not graded yet. Grading starts when the exam run completes.

Nemotron 3 Super 120BScore —

Several critical safety and code violations are present in this scenario:

  1. Torching directly onto plywood deck: Modified bitumen cap sheets must be installed over an approved underlayment or base sheet (typically a fiberglass or polyester-reinforced modified bitumen base sheet or a fire-resistant underlayment). Torching directly onto 5/8" plywood violates manufacturer installation instructions and IBC Section 1507.11.2 and IRC R905.11.2, which require compliance with the roof covering manufacturer’s approved application instructions. Plywood is combustible and can ignite or smolder under torch heat, creating a fire hazard.
  1. Torching up to stucco wall and skylight curb: Torch-applied membranes should not be applied directly against vertical surfaces like stucco walls or wood-framed curbs without proper flashing details and heat protection. This risks igniting combustible framing (especially the wood skylight curb) or damaging the stucco. NFPA 241 (Standard for Safeguarding Construction, Alteration, and Demolition Operations) and IFC Chapter 33 require hot work precautions, including fire watches and clearance from combustibles.
  1. No fire watch after work: Leaving the site at 4 pm with no fire watch after torch-applied roofing work is a serious violation. IFC 3304.5 mandates a minimum 1-hour fire watch after hot work operations when combustible materials are present. Smoldering embers in plywood or debris can ignite hours later.
  1. Lack of thermal barrier: Torch-applied membranes over wood decks typically require a thermal barrier (e.g., gypsum board or approved fire-retardant-treated plywood) unless the assembly is specifically tested and listed for direct application (e.g., UL Class A assembly). Direct torch application to untreated plywood is not permitted under most product listings.

Correct practice: Install a approved base sheet or underlayment first, maintain proper clearances from walls and curbs with metal flashing, use low-temperature torching techniques where needed, and enforce a post-work fire watch. If the crew lacks training in torch safety, this work should be halted and performed only by licensed, certified roofers following manufacturer and fire safety codes.

Bottom line: This is unsafe, non-compliant, and poses a significant fire hazard. Stop work immediately and consult a licensed roofing contractor or fire marshal.

657 tokens · 5.2s

Judge rubric · pending

Not graded yet. Grading starts when the exam run completes.