Gmail Spam Feedback Loop (FBL) for ESPs
Gmail is rolling out the Feedback Loop (FBL) program pilot for ESPs/Bulk senders to help them with spam/abuse detection at the source and identify bad actors exploiting their systems. To protect user privacy, this feedback will at best contain aggregate data that cannot be attributed or traced to a particular recipient. Gmail discussed with various ESPs to understand how best Gmail could address their feedback requirements while respecting user privacy, the most agreed-upon solution was to provide aggregated spam statistics per customer and/or per campaign. Thus, the FBL will report the percentage of user spam markings per campaign and/or per each customer of an ESP for a given day. The purpose of the FBL is purely to help ESPs with identifying spammers/outliers in their traffic and is not meant to assist with deliverability and/or delivery evaluation. The expectation is that the data should be used only for spam and abuse prevention.
Implementation Details:
ESPs will need to embed a header consisting of parameters (called Identifiers) that uniquely identify their customers and/or campaigns for the traffic that they wish to receive the feedback data. Gmail would aggregate and send out feedback reports based on these identifiers.
The header should be in the format:
Feedback-ID: a:b:c:ESPid
- Feedback-ID is the name of the header to be embedded.
- a, b, c are (optional) fields that can be used by the ESP to embed identifiers of their choice (campaign/customer/other). These can be at most 3.
- ESPid is a (mandatory) unique identifier (of length 5 to 15 characters) chosen by the ESP and should be consistent across the mail stream.
The aggregate data will be generated for the first 4 fields (as separated by ‘:’) of the Feedback-ID, starting from the right-hand side. Thus, in the absence (or excess) of a given field, the data will be generated for the rest (except in the case of ESPid – in the absence of which, no data will be generated).
To prevent spoofing of the Feedback-ID by spammers, traffic being sent to Gmail needs to be DKIM signed by a domain owned by the ESP, after the addition of this header. This will be over and above any previous DKIM signing by the ESP’s customers.
A maximum of 10 such unique DKIM (d=) domains may be used across the ESP’s mail stream. Alternatively, the ESP can use multiple subdomains from the same domain(s) as well.
ESPs should ensure that all of their outgoing mail has only one such verified header and overwrite any that might be present already.
Further, the ESPs will have to publish the IPs from which they are sending mail in the SPF records of their signing domains as well – this would also prevent possible issues with the IP list going stale or IPs being relinquished. The sending IPs must have PTR records and resolve to a valid hostname (preferably one of the DKIM domains).
When generating the FBL report, data would be aggregated across the published IPs.
In order to prevent any potential abuse of the system, by way of campaigns having just a single mail or a few emails each, an FBL report will be generated only if a given Feedback-ID Identifier is associated with greater than a certain number of emails, distinct recipients, and user spam reports in a given day’s traffic.
The FBL data will consist of the percentage of user spam markings for each qualifying field(s) in the Feedback-ID, aggregated across all emails received from the ESP on a given day.
An FBL report, consisting of a CSV attachment, will be sent over email (when applicable) daily to an address of the ESP’s choice. The report will pertain to the ESP’s traffic received by Gmail on the previous day.
The FBL data will be generated only for gmail.com recipients (and NOT for recipients on Google Apps or other Google domains).
Appendix:
FBL data will be aggregated by way of each identifier independently and NOT grouped across identifiers, i.e., we will be reporting the spam percentages across all the mails containing a given identifier, irrespective of the position of the identifier in the header.
This is mainly for 3 reasons:
- To keep the fields a, b, c in
Feedback-ID: a:b:c:ESPidopen for the ESP to assign any identifiers of their choice and not be restricted by any particular order that we specify for the sake of grouping. - Allow the use of a limited number of identifiers, i.e., something like
Feedback-ID: a:b:ESPid. - Allow the use of identifiers that are unrelated to each other.
So, for a given day’s traffic, the ESP should ensure that the identifier namespace is unique across fields so that data is not aggregated on unrelated identifiers. For example, an identifier (say a1) used for a CustomerID should not be re-used as CampaignID within the same day’s traffic to ensure that data is not aggregated by unrelated/wrong keys.
If there is a concern about how the identifier namespace can be kept unique or if the preference is for the data to be grouped between two identifiers, the hash of one identifier can be appended to the other, per use case. For example, if the CustomerID is a1 and the Campaign number is 3, a unique identifier a1_3 can be used as a CampaignID.
Also, when choosing identifiers, an ESP should avoid selecting a parameter that will be unique across every single mail (like a unique message ID), as there will be no scope for aggregation on that field.
Below is an example of a Feedback-ID header for illustration:
Feedback-ID: CustomerID2:CampaignIDX:MailTypeID3:ESPid
where
- CustomerID2 is a unique customer identifier.
- CampaignIDX is a campaign identifier and is unique across the board (i.e., no two customers share the same campaign ID).
- MailTypeID3 is an identifier for the nature of the mail (e.g., offers/newsletters/product-update mails, etc.) and can be unique to a customer. Alternatively, in case the ESP would like to measure the spam rate for that mail type throughout their traffic, they can simply keep this identifier common across customers.
- ESPid is the ESP’s unique identifier and can be used for overall stats.
In the above case, we will be sending the spam percentages for each of the 4 identifiers independently (provided they meet the qualifying criterion – as mentioned in the previous section).
Next Steps:
Once you are implementation-ready, please use the confirmation form at (email me for link)to send us the details of your DKIM domain(s) (i.e., the domain in d=), ESPid (of your choice), and the designated email address for the FBL reports to be sent. On receiving these details through the form, we will onboard you in about a week and send you a confirmation email. You will then start receiving FBL reports, whenever there is sizable spam in your traffic.
Note: Do not fill out the form until you are ready with your implementation. Once you enter the data through the form, it cannot be modified – so please be sure to enter the correct details.
In case you have any questions, please refer to the FAQs here. Almost every implementation-related question you may have should be covered in the FAQs.
Our expectation is that you will be acting on the bad actors reported through FBL and prevent them from sending spam in the future.
What is an Identifier?
An Identifier is a key by which you’d like the spam rate aggregated for your FBL report. Examples of Identifiers are: CustomerID_22, CampaignID_67, MailCategoryID_3, etc.
Is it Feedback-ID or X-Feedback-ID?
As you might know, the X in an X-header stands for experimental. The early testers for the Gmail FBL were asked to use X-Feedback-ID since the FBL was an experimental feature back then – but that is not the case anymore.
So, for all current/newer FBL implementations use ONLY Feedback-ID.
What is the ESPid? Do we get to choose our own ESPid or are we assigned one?
ESPid is a unique identifier for each ESP. You can choose an ESPid of your choice. However, when choosing an ESPid, please choose something that is descriptive, 5 to 15 characters long, and contains at least a few letters – ideally the name of your ESP.
Why do we need to DKIM sign our traffic to be eligible for FBL?
This is important for us to correctly identify and aggregate mail coming from your ESP and prevent any spoofing.
My customers are already signing their mail with their own DKIM domains. What do we do?
All you need to do is simply re-sign your traffic after adding the Feedback-ID header with a DKIM (d=) domain owned by your ESP. You have the option to use up to 10 such unique domains to sign your traffic. Gmail supports multiple DKIM signatures.
We have heard that a particular MTA vendor does not support double DKIM signing, an FBL requirement?
Please check with your MTA vendor directly rather than assume. Most vendors have already enabled support for double DKIM, making it simple and seamless to implement the Gmail FBL.
As an ESP, we already sign all our traffic with a DKIM domain owned by us. Do we still need to resign with another domain?
No. All you need to do is add the Feedback-ID before DKIM signing.
Our whitelist customer(s) has an issue with the idea of our ESP re-signing their mail with our DKIM domain, which is an FBL requirement. What do we do about it?
If you feel that a customer of yours might have an issue and cannot be convinced otherwise, you may leave them out and tag the rest of your traffic for the FBL. If you have given a particular customer of yours a “whitelist” status, they must be trustworthy enough indeed for you. Moreover, the Gmail spam FBL is primarily aimed at exposing spammers and outliers in your traffic – an impeccable customer might never even get flagged. You could still use a method like List-Unsubscribe to gather data for such “whitelist” senders.
Why is there an option to sign traffic with up to 10 different DKIM domains?
A lot of ESPs already have an existing setup to sign each category/tier of their traffic with a different DKIM domain. In order to make the Gmail FBL implementation process as less disruptive as possible for them, we have offered the option to use up to 10 distinct DKIM (d=) signing domains.
Do we have to necessarily sign our traffic with 10 different DKIM domains?
No. The upper limit on the number of unique domains that can be used to DKIM sign mail across your traffic is 10.
Can we sign with subdomains of our chosen DKIM domains? If yes, up to how many unique subdomains can we use?
Yes indeed, you can use subdomains of your designated DKIM domains to sign your mail. While there is no upper limit on the number of subdomains, the subdomains should all come from the (at most) 10 unique domains.
What about the selector? Do we need to use a particular selector (s=) if/when double DKIM signing traffic, with a domain owned by us (the ESP)?
The value of the selector (s=) or the lack of it does not matter.
Aside from using a domain owned by our ESP to DKIM sign mail, is there anything different that we need to do in our current DKIM signing process?
Please make sure to add the Feedback-ID header before you (re)sign with your ESP DKIM domain (and sign the Feedback-ID header as well). Nothing else changes – just DKIM sign in your usual way.
Can we use more than 3 Identifiers, other than the ESPid?
No, the maximum number of Identifiers you can use is 3 (excluding the mandatory ESPid).
What if we have less than 3 Identifiers?
That’s not a problem. Just include only your chosen Identifiers in the Feedback-ID. For example: Feedback-ID: a:ESPID or Feedback-ID: a:b:ESPID are all perfectly valid.
Can we use special characters when naming an Identifier?
Yes, except ‘:’ (i.e., colon) – which is to be used as a delimiter between Identifiers, any other special characters are acceptable.
Why is it recommended that we keep the Identifier namespace unique? Can we have Identifiers that overlap?
Since data is aggregated by each unique Identifier (irrespective of its position in the Feedback-ID), for a given day’s traffic, you should ensure that the Identifiers (across fields) are unique and not repeated, so that data is not aggregated across unrelated Identifiers.
For example, Feedback-ID: a1:b1:a1:ESPid will result in data being aggregated on the identifier a1 twice and result in erroneous reports.
Is the data grouped across Identifiers?
No. Data is only aggregated per Identifier and not grouped across. For example, if 3 different spam mails in your traffic had the below Identifiers:
Feedback-ID: a1:b1:c1:ESPidFeedback-ID: b1:ESPidFeedback-ID: c2:b1:ESPid
The count for b1 would be 3, given that it occurs thrice (irrespective of its position in the Feedback-ID).
Should we include all the IPs from which we are sending traffic to Gmail in the SPF records of all the DKIM domains we are using to sign our traffic?
Yes. Please make sure that your SPF records are up to date with all your sending IPs.
What if we decide to send from newer IPs?
Just update your SPF record with the newer IPs and continue to sign the traffic with your designated DKIM domain(s) as usual.
Is there a timeline for when we should be implementation-ready?
While we do not have a strict deadline, the sooner the better.
How long will it be, after we are enrolled, before we start receiving reports?
You should start to receive FBL reports once there is a sizable amount of user-reported spam for any given Identifier in your traffic.
Will we receive FBL reports every day? Will we receive FBL reports for every Identifier that we are using?
You will receive an FBL report as and when there is a sizable amount of spam corresponding to a given Identifier.
What is the Spam_rate column seen in the FBL CSV report?
The Spam_rate is the percentage of user spam markings over all mail delivered to the Inbox (across tabs).
How do we interpret the spam_rate?
The Gmail spam FBL has been designed to report only spammers and outliers. It is safe to assume that anything that gets reported in the FBL, irrespective of the spam rate, is a cause for concern and has the potential to disrupt the deliverability of the rest of your email. So, please investigate and take action upon everything reported – ensure that the spam comes to a stop.
What is the time span over which Gmail FBL aggregates data for each report?
The FBL report that you receive on a given day pertains to your traffic from the previous day (that was tagged with the Feedback-ID header). All user spam markings (for the previous day’s traffic) received till the time of the report generation are counted to calculate the spam_rate.
Help! Our ESP has implemented the FBL and filled out the form. We have not yet received a single report?
Remember that you will receive an FBL report only when there is a sizable amount of spam in your traffic that has received user spam markings. This is important to keep FBL reports noise-free, aside from other reasons.
- Are you signing (and tagging with Feedback-ID) all your Gmail traffic? We have observed that it takes at least 80% of (Gmail) traffic to be signed before ESPs start receiving FBL reports.
- Do your identifiers each correspond to enough mail volume? It may take a given Identifier to be present in a sizable volume of traffic to get sufficient user spam markings. So, make sure your Identifiers have enough volume to each of them.