SQL query to compare two tables and find records without matches

I have two tables, Table1 and Table2. I want to select distinct rows in Table2 that are not contained in Table1. Here is an example:

  Table1        

   | A  |   | sem|
   ------------------
1. | aa |   | 1 |
   ---------------
2. | bb |   | 1 |
   ----------------
3. | aa |   | 2 |
   -----------------
4. | cc |   | 2 |
   ---------------

Table2

   | B  |
   ------
1. | aa |
   ------
2. | aa |
   ------
3. | bb |
   ------
4. | cc |
   ------
5. | cc |
   ------
6. | ee |
   ------
7. | ee |
   ------

I would want the those row who is not common in this two table when sem = 1 like output for sem = 1 result

| B |

  1. | cc |
  2. | ee |

asked Jan 28, 2013 at 2:41

You could try something like this.

SELECT B
FROM Table2 b
WHERE NOT EXISTS (
    SELECT *
    FROM Table1 a
    WHERE a.A = b.B)

From what I can gather, the reason why yours isn't working is because your getting all the values that ARE in both table1 and table2. What you should be doing is like what I did above. Get all the values that are in both tables and check which values in both tables are NOT within the result set, that is why i have set up a sub-query. I'm a little rusty on my SQL so take what i've said with a gain of salt.

answered Jan 28, 2013 at 3:04

user798774user798774

3932 gold badges7 silver badges19 bronze badges

You can do this with an outer join and a where clause. In this case, a right outer join, I think:

SELECT cd.cGenotype
FROM  dbo.TestResults tr right outer join
      dbo.CombinedData cd
      ON (tr.TestResult = cd.cGenotype)
where tr.testresult is null
GROUP BY cd.cGenotype

answered Jan 28, 2013 at 2:44

Gordon LinoffGordon Linoff

1.2m53 gold badges607 silver badges754 bronze badges

1

SELECT DISTINCT b.cGenotype
FROM  dbo.CombinedData b 
WHERE NOT EXISTS (SELECT * 
                  FROM dbo.TestResults a 
                  WHERE a.TestResult = b.cGenotype)

answered Jan 28, 2013 at 2:44

SQL query to compare two tables and find records without matches

Bassam MehanniBassam Mehanni

14.7k2 gold badges32 silver badges41 bronze badges

By:   |   Updated: 2021-10-21   |   Comments (24)   |   Related: 1 | 2 | 3 | 4 | More > Comparison Data and Objects


Problem

Sometimes we need to compare SQL Server tables and/or data to know what has changed. This article shows different ways to compare data, datatypes and table structures when using SQL Server.

Solution

I will cover different methods to identify changes by using various SQL queries as well as a couple development tools.

Let's say we have two similar tables in different databases and we want to know what is different.  Here is a script that creates sample databases, tables and data.

CREATE DATABASE dbtest01
GO
USE dbtest01
GO

CREATE TABLE [dbo].[article] ([id] [nchar](10) NOT NULL, [type] [nchar](10) NULL, [cost] [nchar](10) NULL,
  CONSTRAINT [PK_article] PRIMARY KEY CLUSTERED
  (
  [id] ASC
  ) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO

INSERT INTO [dbo].[article]
  VALUES ('001', '1', '40'),
  ('002', '2', '80'),
  ('003', '3', '120')
GO

CREATE DATABASE dbtest02
GO
USE dbtest02
GO

CREATE TABLE [dbo].[article] ([id] [nchar](10) NOT NULL, [type] [nchar](10) NULL, [cost] [nchar](10) NULL,
  CONSTRAINT [PK_article] PRIMARY KEY CLUSTERED
  (
  [id] ASC
  ) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO

INSERT INTO [dbo].[article]
  VALUES ('001', '1', '40'),
  ('002', '2', '80'),
  ('003', '3', '120'),
  ('004', '4', '160')
GO

The T-SQL code generates 2 tables in different databases. The table names are the same, but the table in database dbtest02 contains an extra row as shown below:

SQL query to compare two tables and find records without matches

Let's look at ways we can compare these tables using different methods.

Compare SQL Server Data in Tables Using a LEFT JOIN

With a LEFT JOIN we can compare values of specific columns that are not common between two tables.

For example, with this SELECT statement:

SELECT *
FROM      dbtest02.dbo.article d2
LEFT JOIN dbtest01.dbo.article d1 ON d2.id = d1.id  

The result set from the LEFT JOIN shows all rows from the left table "dbtest02.dbo.article", even if there are no matches in table "dbtest01.dbo.article":

SQL query to compare two tables and find records without matches

In this example, we are comparing 2 tables and the NULL are displayed if there are no matching rows. This method works to verify new rows, but if we update other columns, the LEFT JOIN does not help.

This can be done both ways to see if there are differences the other way around. This SQL statement will just return the 3 matching rows.

SELECT *
FROM      dbtest01.dbo.article d1
LEFT JOIN dbtest02.dbo.article d2 ON d1.id = d2.id

SQL Server Data Comparison in Tables Using the EXCEPT Clause

Except shows the difference between two tables (the Oracle DBMS guys use minus instead of except and the syntax and use is the same). It is used to compare the differences between two tables. For example, let's see the differences between the two tables:

Now let's run a query using except:

SELECT * FROM dbtest02.dbo.article
EXCEPT
SELECT * FROM dbtest01.dbo.article

The except returns the difference between the tables from dbtest02 and dbtest01:

SQL query to compare two tables and find records without matches

If we flip the tables around in the query we won't see any records, because the table in database dbtest02 has all of the records plus one extra.

SELECT * FROM dbtest01.dbo.article
EXCEPT
SELECT * FROM dbtest02.dbo.article

This method is better than the first one, because if we change values for other columns like the type and cost, EXCEPT will notice the difference.

Here is an example if we update id "001" in database dbtest01 and change the cost from "40" to "1".  If we update the records and then run the query again we will see these differences now:

SQL query to compare two tables and find records without matches

Unfortunately it does not create a script to synchronize the tables.

Compare SQL Server Data in Tables Using the Tablediff Tool

There is a free command line tool used to compare tables.  This can be found in "C:\Program Files\Microsoft SQL Server\110\COM\" folder. This command line tool is used to compare tables. It also generates a script with the INSERT, UPDATE and DELETE statements to synchronize the tables. For more details, refer to this table diff article.

Compare SQL Server Data in Tables Using Change Data Capture (CDC)

This feature is available in SQL Server 2008 and later. You need to enable this feature and you also need to have SQL Server Agent running. Basically it creates system tables that track the changes in your tables that you want to monitor. It does not compare tables, but it tracks the changes in tables.

For more information, refer to these Change Data Capture (CDC) tips.

Compare SQL Server Data Types Between Two Tables

What happen if we want to compare the data types? Is there a way to compare the datatypes?

Yes, we can use the [INFORMATION_SCHEMA].[COLUMNS] system views to verify and compare the information. We are going to create a new table named dbo.article2 with a column with different data type than the dbo.article table:

USE dbtest01
GO

CREATE TABLE [dbo].[article2](
 [id] [int] NOT NULL,
 [type] nchar(10) NULL,
 [cost] nchar(10) NULL,
 CONSTRAINT [PK_article1] PRIMARY KEY CLUSTERED 
(
 [id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO

The difference is that the id is now an int instead of nchar(10) like the other tables.

The query to compare data types of tables article and article1 would be:

USE dbtest01
GO

SELECT
  c1.table_name,
  c1.COLUMN_NAME,
  c1.DATA_TYPE,
  c2.table_name,
  c2.DATA_TYPE,
  c2.COLUMN_NAME
FROM      [INFORMATION_SCHEMA].[COLUMNS] c1
LEFT JOIN [INFORMATION_SCHEMA].[COLUMNS] c2 ON c1.COLUMN_NAME = c2.COLUMN_NAME
WHERE c1.TABLE_NAME = 'article'
  AND c2.TABLE_NAME = 'article2'
  AND c1.data_type <> c2.DATA_TYPE

The results are as follows:

SQL query to compare two tables and find records without matches

The query compares the data types from these two tables. All the information of the columns can be obtained from the [INFORMATION_SCHEMA].[COLUMNS] system view. We are comparing the table "article" with table "article2" and showing if any of the datatypes are different.

Compare if there are Extra Columns Between SQL Server Database Tables

Sometimes we need to make sure that two tables contain the same number of columns. To illustrate this we are going to create a table named "article3" with 2 extra columns named extra1 and extra2:

USE dbtest01
GO

CREATE TABLE [dbo].[article3](
 [id] [int] NOT NULL,
 [type] nchar(10) NULL,
 [cost] nchar(10) NULL,
 extra1 int,
 extra2 int
)

In order to compare the columns I will use this query:

USE dbtest01
GO

SELECT
  c2.table_name,
  c2.COLUMN_NAME
FROM [INFORMATION_SCHEMA].[COLUMNS] c2
WHERE table_name = 'article3'
  AND c2.COLUMN_NAME NOT IN 
     ( SELECT column_name
       FROM [INFORMATION_SCHEMA].[COLUMNS]
       WHERE table_name = 'article'
     )

The query compares the different columns between table "article" and "article3". The different columns are extra1 and extra2. This is the result of the query:

SQL query to compare two tables and find records without matches

Compare SQL Server Tables in Different Databases

Now let's compare the tables in database dbtest01 and dbtest02 with UNION logic (Learn the difference between UNION and UNION ALL) and a subquery with NOT IN logic. The following query will show the different tables in dbtest01 compared with dbtest02:

SELECT 'dbtest01' AS dbname, t1.table_name
FROM dbtest01.[INFORMATION_SCHEMA].[tables] t1
WHERE table_name NOT IN 
     ( SELECT t2.table_name
       FROM dbtest02.[INFORMATION_SCHEMA].[tables] t2
     )
UNION
SELECT 'dbtest02' AS dbname, t1.table_name
FROM dbtest02.[INFORMATION_SCHEMA].[tables] t1
WHERE table_name NOT IN 
     ( SELECT t2.table_name
       FROM dbtest01.[INFORMATION_SCHEMA].[tables] t2
     )

SQL query to compare two tables and find records without matches

Compare schemas using SSDT

SQL Server Data Tools allows to compare schemas of two different tables using the SQL Server Database Project. It can generate scripts that will allow you to synchronize data with some few clicks. Let's take a look to it.

1 - In the database project, go to the Solution Explorer and right click on the database and select the Schema Compare option to compare the tables:

SQL query to compare two tables and find records without matches

2 - In the Select Target Schema, press the Select Connection to select the table destination to be compared with the table in the source. You will be able to select an existing connection or create a new one:

SQL query to compare two tables and find records without matches

3 - Next, press the compare button and it will show the difference. It will show the tables to add or the tables to remove:

SQL query to compare two tables and find records without matches

4 - The tool will show you the entire T-SQL script that you can apply or modify according to your requirements in the tables:

SQL query to compare two tables and find records without matches

SSIS Lookup option

This option is really popular in ETLs (Extraction Transformation Loads). The Lookup is an SSIS transformation task that allows to lookup data using joins using a dataset. The lookup allows you to detect changed data between 2 tables. In the following example, I will compare the tables.

1 - In the SSIS project, double click the Data Flow Task to create it in the Control Flow in order to create a Data Flow sequence:

SQL query to compare two tables and find records without matches

2 - In the Data Flow, drag and drop the OLE DB Source with the source table to compare, the Lookup task and 2 OLE DB Destination tasks:

SQL query to compare two tables and find records without matches

3 - In the Lookup you can fully store the information in cache or connect partially. We also have the no cache option. The full cache mode option will store the data in the SSIS cache. The partial cache will only store on cache values as each distinct value is found in the data flow. Each distinct value will be obtained from the specific source:

SQL query to compare two tables and find records without matches

4 - The task will allow you to define the keys to compare and the lookup columns with the output alias used:

SQL query to compare two tables and find records without matches

As you can see, the Lookup allows to lookup and compare data. You can then get what data was changed between 2 tables. 

Third Party Tools

There are great tools to compare tables including data and schemas. You can use Visual Studio or use other SQL Server Comparison tools.

Next Steps
  • There are multiple tools and ways to compare data and schemas.  Select the method that works best for your needs based on the above queries.
  • For more information refer to these links:
    • Compare SQL Server Datasets with INTERSECT and EXCEPT
    • SQL Server Left Join
    • SQL Server Change Data Capture
    • SQL Server Tablediff Utility
    • Using MERGE in SQL Server to insert, update and delete at the same time
    • SQL Server Cursor Example

About the author

SQL query to compare two tables and find records without matches
Daniel Calbimonte is a Microsoft SQL Server MVP, Microsoft Certified Trainer and Microsoft Certified IT Professional.

View all my tips

Article Last Updated: 2021-10-21


Comments For This Article

Friday, February 19, 2021 - 6:39:38 AM - Peter Vella Back To Top (88265)
Great Cmd line., The problem with other tools is that the tables have to be indexed. It would be great if the above would let us exclude columns.

Monday, May 13, 2019 - 8:52:21 AM - db042188 Back To Top (80057)

 pls use caution with the except statement.  it drops duplicates automatically from the left and right.  in biz intelligence that isnt acceptable as the dupes are either necessary or an error that needs to be caught.


Friday, May 10, 2019 - 1:23:28 AM - testMail Back To Top (80013)

I have done this using powershell and you can also use this :

$ss = "souce_server" ;
$sd = "source_database" ;
$st = "source_table" ;

$ds = "destination_server" ;
$dd = "destination_database" ;
$dt = "destination_table" ;

$tablediff_directory = "C:\Program Files\Microsoft SQL Server\130\COM\" ;
$op = "D:\test.sql" ;

CD $tablediff_directory

tablediff.exe -f $op -sourceserver $ss -sourcedatabase $sd -sourcetable $st -destinationserver $ds -destinationdatabase $dd -destinationtable $dt

Wednesday, February 14, 2018 - 6:17:35 AM - Ranjan Back To Top (75213)

Thanks Daniel for this article. I have a requirement to find the difference between two tables of same data but the primary keys will be different, so I have a difficulty in identifying which all records are inserted in the source records and which all records are updated in the source table. Could you please help me?

I tried MERGE and EXCEPT, but not much luck.

Thanks,

Ranjan


Wednesday, September 20, 2017 - 3:18:04 AM - Muhammad Waseem Back To Top (66436)

Dear Daniel

Its amazing article you wrote. I am sure it will help others  as well ,  Welldone !!!!!


Tuesday, October 18, 2016 - 3:52:16 PM - Manjinder Back To Top (43589)

Thank you very much Daniel! Well written & easy to understand and follow plus thorough as most articles are on this site. Keep up the excellent work of sharing the knowledge! :-) 


Wednesday, February 17, 2016 - 12:54:47 PM - Barry Seymour Back To Top (40704)

THANK YOU THANK YOU THANK YOU! I never knew of the EXCEPT keyword. It makes a ton of my earlier attempts at table comparison moot, but I don't care! An excellent summation.


Wednesday, January 28, 2015 - 2:28:03 AM - PRADEEP KUMBHAR Back To Top (36081)

when i run query in sql server 2012 it join 2 tables but all field contain same data display.

Please get the solution .


Tuesday, July 29, 2014 - 10:38:59 AM - Greg Robidoux Back To Top (33918)

Hi Jayesh, this can be done but it takes a lot of coding to figure out which rows are different and then which columns are different.  You could use the EXCEPT method shown above to show records that are different, but then you will need to take the results from that and write a process to compare column by column.  I don't think we have a tip that does this, so maybe someone could write a tip.


Tuesday, July 29, 2014 - 8:05:10 AM - Jayesh Back To Top (33913)

Helllo please read this instead of above,

i want to compare two tables from different database, suppose

both tables have 5 raw

in which

table 1 and table 2 has 4 raw exactly same,

but fifth raw has minor change in one column value

if i want to to see only that value not entire raw then

what to do?

red gate can't give specic difffrent column value


Thursday, July 10, 2014 - 4:45:38 AM - Nitin Sharma Back To Top (32623)

why do we use procedure


Tuesday, June 10, 2014 - 6:46:58 PM - Arun kumar (SQL Developer) Back To Top (32177)

I liked this post, but this will not full fill for the dependent table. I have faced an issue when accidently 1 master table updated in source table that has updated multiple table on multiple server, in some table it uses foreign key relation ship. Now we have to identify all the changes related to this one. This changes also replicated to diffrent server. We were thinking can we do with this approach but did not found solution on the spot. We have worked 1 by one table then we sync the table.

If you developer think it can also generate script for dependent table please share. Thanks a lot !!!!


Wednesday, November 6, 2013 - 5:06:40 PM - Salvador Nava Back To Top (27433)

Thanks a lot!!! Very useful!!!


Thursday, August 29, 2013 - 2:50:55 AM - Geoffrey Back To Top (26520)

I recomment SQL Negotiator Pro from Aphilen software www.aphilen.com they offer a free version which may suffice your needs

The full blown version comes with the ability to

Compare unlimited databases, generate management reports, graphically display database dependencies, SQL server monitor etc etc



Tuesday, March 26, 2013 - 7:35:23 AM - Satish Back To Top (23013)

Thank you Daniel for your good article. 


Sunday, March 17, 2013 - 3:54:11 PM - Ari M�enp�� Back To Top (22836)

Excellent page, I have found this already twice when needed this! Maybe I should now bookmark this.

Thank you!


Tuesday, October 23, 2012 - 11:27:52 AM - Boris Tyukin Back To Top (20052)

@David McKinney

sorry, looks his site is down...but when I tested this query below, it was the fastest:

SELECT MIN(id) AS id, col1, col2, col3 FROM (SELECT id, col1, col2, col3 FROM tbl1 UNION ALL SELECT id, col1, col2, col3 FROM tbl2) X GROUP BY col1, col2, col3 HAVING COUNT(1)=1

It also works well with NULLs. He does Union ALL, but HAVING COUNT(1)=1 will output only records not found in one of the tables, so matches will be suppressed


Tuesday, October 23, 2012 - 10:58:06 AM - David McKinney Back To Top (20051)

@Boris I couldn't get your link to work (for Lee Everest's article).

I have a method using UNION as well but not UNION ALL

It relies on the fact that

select 'alpha' UNION select 'alpha' will only return 1 row (and not 2).  Hence if you UNION 2 tables with 1000 rows each together, if they're identical, the result will have 1000 rows also.  You can then look for duplicates on the PKey field(s) to identify those rows which are different.


Tuesday, October 23, 2012 - 9:04:26 AM - Daniel Calbimonte Back To Top (20050)

Yes,

There is already a pretty nice article related to third party tools to compare data

http://www.mssqltips.com/products.asp?catid=25

Personally I love SQL Data Compare from Redgate and SQL Data Compare from devart.

My original article included some demos about 3 third party tools, but the editors decided not to promote third party tools.


Monday, October 22, 2012 - 3:56:53 PM - John Fox Back To Top (20042)
A simple method that I use that works quite well and is free is to create a SQL query that looks like the following. If one is familiar with tools that use regular expressions, it can be done quite quickly by starting with the output of 'script table as select' using SMS. Assuming the keys are the primary keys, it is only necessary to check one of them for each table to null to detect a 'missing row' condition. And it is amazing what data types compare quite well using '-1' as the default if null. Of course, one has to be smart and if '-1' is a valid data condition then either some other value of checking specifically for 'is null' should be used. After running the SQL, I usually load it into excel, cut all of the data from 'Test' to the last column and last row, then paste it after the 'prod' data. A quick sort by key1/key2 then makes it pretty easy to walk through the data.

select 'Prod',t1.key1, t1.key2, t1.field3, t1.fiel4, 'Test',t2.key1, t2.key2, t2.field3, t2.fiel4
from ProductionTable t1
full join TestingTable t2 on t1.key1 = t2.key1 and t2.key2 = t2.key2
where t1.key1 is null
or t2.key1 is null
or isnull(t1.field3,-1) != isnull(t2.field3,-1)
or isnull(t1.field4,-1) != isnull(t2.field4,-1)


Monday, October 22, 2012 - 2:02:42 PM - TimothyAWiseman Back To Top (20041)

Great article, thank you for providing it.

I somewhat agree wtih Marting Borgars.  I am a big fan of Red Gate's SQL Data Compare and Red Gate's SQL Compare when looking at schemas, I have been extremely happy with them.  

But there are times when those won't be available.  As Boris points out, they aren't free (well worth the price though), so you may not always have them, especially when working for a small client on a small project.  Some institutions also have very lengthy validation procedures for installing software due to security concerns. 

It is also good to be able to do it "by hand" just for the sake of having a thorough knowledge base and understanding of how things work, even if you will use the tools most of the time. 


Monday, October 22, 2012 - 10:14:23 AM - Boris Tyukin Back To Top (20038)

Thanks for the interesting article!

Red Gate's SQL Data Compare tool is nice indeed and we use it at work but it is not free so it is great to see alternatives.

Some time ago I bookmarked very fast and cool method, described by Lee Everest here

The idea is to use query like that:

SELECT MIN(id) AS id, col1, col2, col3 FROM (SELECT id, col1, col2, col3 FROM tbl1 UNION ALL SELECT id, col1, col2, col3 FROM tbl2) X GROUP BY col1, col2, col3 HAVING COUNT(1)=1



How do you get not matching records from two tables in SQL?

This query can solve the problem:.
( SELECT id FROM orders1 EXCEPT SELECT id FROM orders2 ) UNION ( SELECT id FROM orders2 EXCEPT SELECT id FROM orders1 ).
SELECT id FROM ( SELECT DISTINCT id FROM orders1 UNION ALL SELECT DISTINCT id FROM orders2 ) AS temp_tbl GROUP BY id HAVING COUNT(*) = 1..

How can I get matched and unmatched records from two tables in SQL?

To get all of the rows from just one of the tables – the matched rows as well as the unmatched rows – you need to use the LEFT JOIN or the RIGHT JOIN .

How do I find different records in two tables in SQL?

sql query to return differences between two tables.
SELECT DISTINCT [First Name], [Last Name], [Product Name] FROM [Temp Test Data] WHERE ([First Name] NOT IN (SELECT [First Name] ... .
SELECT td.[First Name], td.[Last Name], td.[Product Name] ... .
SELECT [First Name], [Last Name].

How can I get mismatched data from two tables in SQL?

Find Mismatched Data between SQL Server Tables using LEFT JOIN, EXCEPT, NOT IN and NOT EXISTS.
Creating the Dataset. ... .
Compare AllNumbers and MissingNumbers Tables. ... .
1 - Find Missing Numbers Using LEFT OUTER JOIN. ... .
2 - Find Missing Numbers Using EXCEPT. ... .
3 - Find Missing Numbers Using NOT EXISTS..