Starting with version 2.5, Upsource doesn't send notification emails as soon as some event takes place but aggregates all events that happened within a certain time frame and combines them into a single email. The default aggregation interval is 5 minutes. If you feel that you're getting too many emails anyway, you can increase that interval.
1. Go to the <Upsource_home_directory>/conf folder
2. Open the upsource.user.properties file (create it if it doesn't exist).
3. Put the following line in it:
where X is the aggregation timeout in milliseconds, e.g. 600000 is 10 minutes.
4. Restart Upsource.