Checking for Duplicate Records

Looking for and removing Duplicate Records.

Background. In the early stages of the website, there was potential to add duplicate records from the wildlife diary. This arose because some records were added by the web designer and also because Mammal records were entered separately in the early stages to meet ERCCIS requirements. Some Data Analysts were unaware of this when adding the complete set of members diary records at a later date. These have now mostly been removed, but errors can occur in adding the diary records and it is important to check for this.

The following links provide a simple way to check, year by year. A script reports observations where an observer has reported observing a species several times in the same quarter and year, but has produced several separate species reports rather than one report noting several sightings. This may indicate a duplicate entry.  

The page for this facility for the following years are: 

Duplications Checks complete:  2002  2003 2004  2005 2006 2007 
Ongoing:  2008 2009 2010 2011 2012 2013

Each page opens in a new window, which can be closed down to return to the original page. If no results show on a page, then there are no duplications For other years, it is necessary to create a new page and edit the script to change the year, as shown below.

Creating pages for other years.  

Open any of the existing pages in edit mode. Copy the whole contents of the body.
Create a New Page through Create Content
Give the new page it's new Title.
At the bottom of the body box, change the Input Format to PhP Code, then Submit before pasting into the box.
Select Edit again and paste the contents copied from the other page into the box.
Edit the line WHERE observation.obs_year='2005' AND node.nid = observation.nid AND node.nid=observation_details.nid AND node.uid=users.uid AND term_data.tid=observation_details.species
to insert the new Year making sure it is between the quotation marks eg '2012'
Submit for the new page to be active.
Note the Url of the new page and edit this note to add the new year and use the chain icon to link it to the Target for new page created. Select Open in a new window in the dialogue box that opens and then Submit

Help notes for using the above duplicate checks.

Open the relevant year.

Note the  species suggested as duplicates for a given Recorder in a given quarter.

Select any one of the possible duplicates and click on the end of the line to show all the record entries for that recorder in that quarter.

Look at each of the record entries of up to 25 species and establish which entry has the duplicates.

If this entry has ALL the duplicates, delete the entry by going to edit and then delete. If there only a few species in this entry that are on the duplicate list, set the count of each of the duplicate species to Zero (0) and submit. Close the page and look at the duplicates for that year again to check that they have been removed.