Legal / Privacy
Privacy at ML Referee.
This notice explains what ML Referee collects, why it is processed, when it is disclosed, how long it is retained, and the choices available to you.
Who is responsible
ML Referee is operated by Sandra Carrico as an individual. Sandra Carrico is the controller of personal information processed to operate ML Referee. Use the private support form for privacy questions and requests to access, correct, or delete information.
What this notice covers
This notice covers the public website, beta and support forms, Auth0 account connection, credit and access administration, submitted evaluation data, evaluation results, and operational records for the hosted ML Referee service.
Account connection and identifiers
ML Referee uses Auth0 for sign-in and authorization. The ChatGPT connection requests the openid, email, and profile identity scopes plus ML Referee read and write scopes. Auth0 and any identity provider you choose may process information associated with those scopes under their own notices.
The ML Referee backend validates the authorization token and uses its Auth0 issuer and subject identifier to create a one-way keyed digest. It stores that digest with a randomly generated tenant identifier so later authenticated requests return to the same private tenant. The backend code does not read or store the name or email claims from the token.
Evaluation data and results
Depending on the evaluation, submitted data may include observation and label-availability times, outcomes or labels, baseline features or scores, candidate or diagnostic features or scores, and optional row, entity, batch, or weight fields. The service also processes the evaluation contract, quote, status, and result.
Evaluation submissions may contain synthetic data, non-personal data, or genuinely anonymized data. Do not submit regulated data or identifiable personal data, including pseudonymized person-level records. Ordinary account and support contact details may be provided through the designated account and support channels, but must not be included in evaluation datasets. Do not submit credentials, payment information, government identifiers, raw private prompts, or other sensitive data.
The current backend instructs users to de-identify data but does not technically detect and reject every prohibited submission. Submitters are responsible for ensuring their evaluation data meets these requirements.
Evaluation inputs, contracts, and results are tenant-private and encrypted at rest with a tenant-derived key. During an evaluation, a restricted worker processes a temporary copy and discards that temporary working directory after the run. Retained service copies follow the expiration and deletion rules below; finishing a run does not instantly delete every copy.
LLM use and your evaluation data
ML Referee does not send evaluation datasets to an LLM as part of running an evaluation. It does not reuse customer evaluation datasets or candidate features to train or fine-tune shared or general-purpose AI models. An evaluation may fit statistical models for your requested comparison; that computation is part of your evaluation, not training a shared AI service.
Your connected AI client is a separate service. If you use ChatGPT, Claude, Copilot, Codex, or another AI client, that client and its provider may process data you give it or make available through tools, including tool arguments and returned results. Their own account settings and policies govern that processing. ML Referee cannot promise that no LLM will see data you share through your chosen client.
AI-assisted debugging of evaluation data requires your explicit permission. Before sharing any evaluation data with an AI-assisted support tool, ML Referee will ask you to approve the specific material, tool, and troubleshooting purpose. Opening a support ticket does not grant that permission. Approval for troubleshooting is not permission for ML Referee to reuse your dataset for shared-model training. Do not include confidential evaluation data in an initial support request; ordinary support correspondence is handled by the form and mailbox providers described below.
Why information is processed
ML Referee processes account and service information to authenticate requests, isolate each tenant, review and administer beta access, maintain credit balances, validate and run authorized evaluations, return results, prevent abuse, troubleshoot failures, respond to support and privacy requests, and meet legal obligations.
Legal bases for processing
Where European or United Kingdom data-protection law applies, ML Referee processes information as needed to take steps you request before providing the service and to perform the service agreement; for legitimate interests in operating, securing, measuring, and improving the service and responding to genuine requests; to comply with legal obligations; and with consent when the law requires it. You may object to processing based on legitimate interests or withdraw consent without affecting earlier lawful processing.
Evaluation retention and deletion
The submitting user or agent selects the encrypted object-retention window. Standard uploads may be retained from 60 seconds to 24 hours and default to 24 hours. Large campaigns and repair evaluations require enough time to finish, currently at least 4,200 seconds, and default to two hours. The evaluation contract and report expire with their associated input; job completion does not start a new retention period.
The worker checks for expired objects and removes the encrypted input, contract, and report after the selected window. Physical deletion can lag while an evaluation is processing or if the cleanup worker or service is unavailable. An authenticated tenant may also use the service’s deletion tools to delete a retained upload or an audit’s retained input, contract, and report before expiry.
Deleting evaluation objects does not currently delete every associated operational record. ML Referee stores pseudonymous tenant and identity mappings, beta requests and grants, credit-journal entries, quote and job metadata, opaque object references, service metrics, worker timing and usage records, and deletion or security events. These records currently have no automatic deletion deadline, and some are append-only. Application-level aggregate metrics do not include submitted datasets, feature names, prompts, or report content, but some records contain opaque tenant, job, quote, or object identifiers.
Report verification records
When ML Referee issues an evaluation report, it creates SHA-256 fingerprints of the public result and report text. We retain these fingerprints with pseudonymous account, audit, and report identifiers, the issuance timestamp, report-format version, and byte lengths to verify report integrity and investigate disputed reports.
These verification records contain no report text or evaluation dataset contents. They remain after report contents expire or are deleted. Access is restricted, and the records currently have no automatic deletion deadline. Because they remain associated with pseudonymous account identifiers, we treat them as potentially personal information. Privacy and deletion requests are reviewed individually under applicable law.
A matching fingerprint establishes consistency with the issued artifact, not the correctness or safety of its conclusions.
Operational and provider logs
The deployed application configuration disables the Uvicorn HTTP access log, and the provided Caddy configuration discards its HTTP access log. Operational and error logs still exist. System text logs rotate weekly with four archives, and PostgreSQL text logs rotate weekly with ten archives. The system journal has no explicit time-based retention limit configured. Auth0, the hosting provider, operating system, network, and infrastructure services may also create security or operational logs under their own configurations and policies.
Backups
Backup copies may remain after live evaluation objects are deleted. Backup periods and deletion behavior depend in part on provider and infrastructure settings. Local deployment and configuration backups do not currently have a verified automatic expiry. ML Referee therefore does not promise a specific backup expiration period or immediate removal from every backup copy.
Website traffic and support forms
ChatGPT Sites records aggregate website traffic. The operator’s view shows totals such as unique visitors, page views, requests, paths, and activity over time; it does not provide the operator with person-level browsing histories or search-query reports. The website does not add a client-side analytics SDK, advertising identifier, tracking pixel, or tag manager.
Beta and support forms collect the reply email and the information entered in the form. Beta requests also ask for name, company, requested use, candidate scale, and a short description. FormSubmit processes the submission and forwards it to the operator’s Instinct email service. FormSubmit states that it retains submissions for 30 days. Messages delivered to the Instinct mailbox are used to review and respond to the request. A finite deletion schedule covering the mailbox and form-provider copies has not yet been implemented and verified.
Hosting and service providers
The public website is hosted through OpenAI’s ChatGPT Sites service and its delivery network. The evaluation service runs on a dedicated Ubuntu server with PostgreSQL and Caddy on a Hetzner network registered in the United States; the precise physical production region has not been independently verified. Auth0 provides identity and authorization. FormSubmit delivers website forms, and Instinct provides the destination mailbox. These providers process information needed to deliver, secure, and support their parts of the service under their own terms and privacy practices. Auth0’s actual log-retention setting cannot be established from the evaluation server and still must be verified in the Auth0 dashboard.
ML Referee is independent of OpenAI and Auth0. Identifying those providers does not imply sponsorship, endorsement, or partnership.
When information is disclosed
ML Referee discloses information only as needed to operate and protect the service: to hosting, identity, form-delivery, mailbox, infrastructure, and other service providers working on its behalf; when you direct a disclosure; to professional advisers who need it for legal, accounting, security, or audit work; to comply with law, legal process, or regulatory requests; to investigate fraud, misuse, or security incidents; or in connection with a proposed or completed transfer of the ML Referee service or business. Evaluation reports are returned to the authenticated tenant and are not made public by ML Referee.
ML Referee does not sell personal information, disclose it to data brokers, or share it for cross-context behavioral advertising. Its limits on using evaluation data for AI training and AI-assisted support are described under LLM use and your evaluation data.
Aggregated and deidentified information
ML Referee may use aggregated or deidentified operational information to measure, secure, troubleshoot, and improve the service. It will maintain that information in deidentified form and will not attempt to reidentify it except to test whether a deidentification method works as intended or when law permits or requires it.
Third-party sites and services
Links and connections may lead to services ML Referee does not control, including ChatGPT, Auth0, FormSubmit, and provider documentation. Their own terms and privacy notices govern their independent processing. A link, connection, or provider reference does not imply endorsement or partnership.
Cookies and browser storage
The public website code does not set analytics, advertising, or preference cookies and does not write to local storage, session storage, IndexedDB, or a service-worker cache. The hosting network may set a security cookie needed to identify likely automated traffic and protect the site. Auth0 and FormSubmit may use their own necessary security and anti-abuse technologies when you sign in or submit a form.
California disclosures
No sale or advertising share. ML Referee does not sell personal information, share it for cross-context behavioral advertising, or offer a financial incentive in exchange for personal information.
No cross-site advertising tracking. ML Referee does not add advertising networks or social-media widgets to these pages. Because the service does not sell information or use it for cross-context behavioral advertising, there is no sale or advertising share for a browser preference signal to opt out of.
International processing
Service providers may process information in the United States and other countries where they operate. When data-protection law requires transfer safeguards, the applicable provider arrangements and legal mechanisms govern those transfers.
Your choices and rights
Depending on where you live, you may have rights to request access, correction, deletion, restriction, objection, or portability, and to complain to a data-protection authority. Use the support form and choose “Privacy or deletion.” ML Referee may need to verify that the request concerns your information before acting on it.
You can also use the authenticated deletion tools for retained evaluation objects. Because no complete account-deletion route currently exists for all pseudonymous metadata, the operator must review requests affecting account, credit, audit, or security records individually. A finite retention schedule for the minimal record of a completed privacy request has not yet been implemented and verified.
Adults only and automated decisions
ML Referee is intended only for adults 18 and older using the service for business or professional research. It does not seek or knowingly collect information from children. If information from a child is discovered, use the support form to request review and deletion.
ML Referee evaluates submitted feature evidence; it does not make legal or similarly significant decisions about individuals. Identifiable and pseudonymized person-level evaluation data is prohibited.
Security and data minimization
Use the minimum data needed for the evaluation or support request. Never send passwords, OAuth secrets, access tokens, recovery codes, or sensitive datasets through a support form. No internet service can guarantee perfect security.
Changes to this notice
Material changes will be posted here with a revised effective date. If a material change affects how previously submitted information is used, ML Referee will provide direct notice when reasonably possible and required by law.