Welcome to the LimeSurvey Community Forum

Ask the community, share ideas, and connect with other LimeSurvey users!

Bounce Processing false negatives

  • 412837
  • 412837's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
6 years 4 months ago #160234 by 412837
Bounce Processing false negatives was created by 412837
Hi!

I am installing LS and just got to test the bounce processing.

I add a participant with a known invalid email, such as invalid-user@mydomain.com.

I can see the bounce email arrive to the pop account for bouce emails, however when I do process bounces the popup reads:

1 unread messages were scanned, none were marked as bounce by the system.You can now close this modal box.



what does: "None were marked as bounce by the system" means?

Is it that postfix is not generating a valid return email?

Is it that LimeSurvey did not detect it as a bounce email?

What could be going on?

How can I debug it?
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
6 years 4 months ago #160239 by DenisChenu
Replied by DenisChenu on topic Bounce Processing false negatives
Hi,

Bounce system track only unread message, maybe you read it before ?

Else, after bounce was done : look at header (show+complete+mail+header in your search tool), bounce system use it to track donw the token and the survey id.
Some server broke totally this header.

If you are sure it broken : copy/paste the header here.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
  • 412837
  • 412837's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
6 years 4 months ago #160337 by 412837
Replied by 412837 on topic Bounce Processing false negatives
I know what is going on now.

The problem is that I expected the bounces to show in the Central Participant Database (where I picked the users from).

I took me a while to find the list of responder of my survey, where it is reflected.

Another problem is that the bounce processing system is not marked my bounce message as read, then after the first time, it keeps saying that 1 message has been processed but it is not a bounce.

I guess it was just a bit confusing, but I think it is working.
The topic has been locked.
  • LouisGac
  • LouisGac's Avatar
  • Visitor
  • Visitor
6 years 4 months ago #160558 by LouisGac
Replied by LouisGac on topic Bounce Processing false negatives
Here the code:
github.com/LimeSurvey/LimeSurvey/blob/ma...admin/tokens.php#L76


the concept is very simple:
1. it open the inbox of the "bounce email address" (the email you provided for that task)
github.com/LimeSurvey/LimeSurvey/blob/ma...dmin/tokens.php#L184

3. it get the unread mails:
github.com/LimeSurvey/LimeSurvey/blob/ma...dmin/tokens.php#L191

4. it choose the mails related to LimeSurvey (with a survey id and a token id in the header )
github.com/LimeSurvey/LimeSurvey/blob/ma...dmin/tokens.php#L200
github.com/LimeSurvey/LimeSurvey/blob/ma...dmin/tokens.php#L205

5. For those mail, it checks if the status is "bounced"
github.com/LimeSurvey/LimeSurvey/blob/ma...dmin/tokens.php#L212

Of course, not all the mail providers reject the email with a "bounced" status. Some of them just reject them silently. There is nothing you can do about that.
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
6 years 4 months ago #160561 by DenisChenu
Replied by DenisChenu on topic Bounce Processing false negatives
Bounce system put READ by default (or delete the email) : github.com/LimeSurvey/LimeSurvey/blob/ma...dmin/tokens.php#L225
Then ONLY email by limesurvey are set as read. Spam for example are not set as READ (or deleted) by LimeSurvey.

Else : false positive can be done by another reason :
- “out-of-office message” because in fact : it's a bounce …
It's the reason you can choose to not delete the email, but control the real final message.

I think the 'long delay' didn't sent a Bounce (can be tested with sending an email to example@example.org).

Denis

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
More
6 years 2 months ago #162275 by snowsam
Replied by snowsam on topic Bounce Processing false negatives
Hello!

I'm running Version 3.0.1+171228 and trying to debug where bounce processing is not recognizing some bounces. I just issued my first real survey, but during testing I had the same issue.

LS is able to log in and read the IMAP emails, but does not report bounces as bounced. So there is some problem with my emails not being in a format LS expects.

During my testing there was one time that processed one bounce email, so I figured I had gotten the setting correct and moved on. However, now I have "live" bounces and it does not work.

Here is a sample header and beginning of the attached email:

Return-Path: <>
Delivered-To: survey-no-reply@laurel-point.net
Received: (qmail 23062 invoked for bounce); 8 Jan 2018 12:32:14 -0000
Date: 8 Jan 2018 12:32:14 -0000
From: MAILER-DAEMON@laurel-point.net
To: survey-no-reply@laurel-point.net
Subject: failure notice

Hi. This is the qmail-send program at laurel-point.net.
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.

<example@example.com>:
148.163.157.120 does not like recipient.
Remote host said: 550 5.1.1 User Unknown
Giving up on 148.163.157.120.

--- Below this line is a copy of the message.

Return-Path: <survey-no-reply@laurel-point.net>
Received: (qmail 23014 invoked by uid 33); 8 Jan 2018 12:31:56 -0000
To: Example <example@example.com>
Subject: TJ SNOW is requesting your input
X-PHP-Originating-Script: 33:class.phpmailer.php
Date: Mon, 8 Jan 2018 07:31:56 -0500
From: TJ SNOW <survey-no-reply@tjsnow.com>
Message-ID: <a8e0506c82a0810fcba732cf22bf311f@survey.tjsnow.com>
X-Mailer: PHPMailer 5.2.22 ( github.com/PHPMailer/PHPMailer )
X-surveyid: 599393
X-tokenid: rovuPDgiB02EqVU
X-Surveymailer: LimeSurvey - TJ SNOW Emailer (LimeSurvey.org)
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="b1_a8e0506c82a0810fcba732cf22bf311f"
Content-Transfer-Encoding: 8bit

This is a multi-part message in MIME format.

--b1_a8e0506c82a0810fcba732cf22bf311f
Content-Type: text/plain; charset=us-ascii


[Logo]

Dear Example,

You have been invited to participate...


I will do more troubleshooting, and report what I discover.

Thanks for posting the links above to the correct areas of the code. That is a big timesaver when starting from scratch to try to figure things out.
The topic has been locked.
More
6 years 2 months ago #162297 by snowsam
Replied by snowsam on topic Bounce Processing false negatives
For reference, I'm using imap with courier-imap as my server.

OK... I've got it figured out.

At github.com/LimeSurvey/LimeSurvey/blob/ma...dmin/tokens.php#L178
there is the line
Code:
$header = explode("\r\n", imap_body($mbox, $sMessageID, FT_UID &amp; FT_PEEK)); // Don't mark messages as read

It should instead be:
Code:
$header = explode("\r\n", imap_body($mbox, $sMessageID, FT_UID | FT_PEEK)); // Don't mark messages as read

That difference in listing the multiple FT Flags was making LS not get anything for the message body, which meant it was never getting a match.

Bug reported at bugs.limesurvey.org/view.php?id=13145

While discussing bounce processing, I have a two quick questions and maybe someone is able to provide insight:

1. I see in the code there is a "bounceremove" variable that would move processed move processed and matched messages to the trash. But I don't see anywhere that variable can get set on the survey level or the main config level. What am I missing?

2. Has there been any discussion that instead of deleting or leaving in the inbox unread, that messages be moved to a "processed" folder? To me, it is easy to mark a message as read by mistake when checking that inbox by mistake. So if I mark one as unread by mistake, then processed deleted in LS, and then grab all the unread and move them to deleted, I just missed processing one. By having the LS process move messages to another folder, this is avoided.

Thoughts?

Thank you,
Sam
The topic has been locked.

Lime-years ahead

Online-surveys for every purse and purpose