ENTITY FRAMEWORK IS AN ABSTRACTION
“Object-Relational Mapping frameworks are a convenient way to provide an abstraction for data
access in an object-oriented application. For .NET applications, Microsoft's recommended O/RM is
the Entity Framework. With any abstraction though, performance can become a concern.”
-- MSDN
Performance Considerations for EF 4, 5, and 6
But depending on who you ask…abstraction is the “fourth’ pillar of Object Oriented
Programming: https://www.lynda.com/C-tutorials/Four-pillars-OOP/604242/634143-4.html
DON’T BLAME ENTITY FRAMEWORK FOR PERFORMANCE PROBLEMS
“Compared to writing your own SQL to access data, you can become miraculously more productive
by using Entity Framework (EF). Unfortunately, several traps that are easy to fall into have given it a
reputation for performing poorly; but it doesn’t have to be this way! The performance of Entity
Framework may once have been inherently poor but isn’t any more if you know where the
landmines are.”
-- Ben Emmett
https://www.red-gate.com/simple-talk/dotnet/net-tools/entity-framework-performance-and-what-you-can-do-
about-it/
CONTROL WHAT WE CAN
“Typically, SQL Server database performance is affected by stress on the following hardware resources:
• Memory
• Disk I/O
• Processor
• Network”
https://www.safaribooksonline.com/library/view/sql-server-query/9781430267423/9781430267430_Ch02.xhtml
We have control over some of these as programmers…
“It doesn't matter how fast we make
the hardware, software programmers will piss it all away eventually".
• Programmers are “off the hook” because their prototype works great and is very fast, but nobody takes the time to stress test
the application with large sets of data.
ITERATIVE VS. SET BASED PROCESSING
• C#/VB and many .NET application are built using iterative programming
• SQL Server is a SET based method of working with data (not RBAR though!)
“Iterative processing is quite common outside of a relational database system; think arrays and
loops. Non relational software systems typically have no SET based functionality - hence the
widespread integration of SQL Server, Oracle, DB2 etc. systems. And I think that this is where the
problem comes in.”
http://www.sqltuners.net/blog/13-04-30/SET_based_processing_vs_Iterative.aspx
ENTITY FRAMEWORK – ADDING, DELETING, AND UPDATING RECORDS: THERE'S LITTLE YOU HAVE TO WORRY ABOUT
“Manual synchronization on single entity is easy task. You just need to attach entity and call
AddObject for inserting, DeleteObject for deleting or set state in ObjectStateManager to Modified
for updating. The real pain comes when you have to deal with object graph instead of single entity.
This pain is even worse when you have to deal with independent associations (those that don't use
Foreign Key property) and many to many relations. In that case you have to manually synchronize
each entity in object graph but also each relation in object graph.”
https://stackoverflow.com/questions/3635071/update-relationships-when-saving-changes-of-
ef4-poco-objects/3635326#3635326
BUT WHAT ABOUT THE R IN CRUD?
“I have clients who need to build very complex queries dynamically. These are cases where their
users have many fields and a variety of options for constructing a search, and in code, we need to
build a query. Although LINQ to Entities is composable and very flexible, there may be a point at
which you begin to hit walls.”
https://www.safaribooksonline.com/library/view/programming-entity-
framework/9780596807276/ch05.html
Not every problem has a perfect solution. However the more you know about the
problem helps and having different approaches to solve a problem helps the most.
“ITS SLOW!” TRY USING SQL SERVER PROFILER TO SEE WHY
“Understanding the queries generated by the ORM
In most applications these days, we use Object-Relational Mapping (ORM), such as Entity
Framework or NHibernate. As you might know, the primary objective of the ORM is to enable you to
write the data access layer using domain-based classes and objects instead of writing queries directly.
However, it does not mean that you never need to understand the basics of the SQL queries generated,
or the optimization of these queries. Sometimes, the generated query from Entity Framework
may not be optimized, so a better practice would be to run the profiler, analyze the
generated queries, and tune them as per your needs. You can use the interceptors in
Entity Framework to log the SQL queries.”
Citation Needed! Please don’t sue me… this course was free.
SQL Query Performance Tuning“….SQL Server instances are being put on virtual machines, either locally or in hosted environments, where the hardware behavior is not
guaranteed. Databases are going to platform as a service systems such as Amazon RDS and Windows Azure SQL Database. You still have to deal
with fundamental database design and code generation. In short, query performance tuning remains a vital mechanism for improving the performance
of your database management systems. The beauty of query performance tuning is that, in many cases, a small change to an index or a SQL query can
result in a far more efficient application at a very low cost. In those cases, the increase in performance can be orders of magnitude better than that
offered by an incrementally faster CPU or a slightly better optimizer.
There are, however, many pitfalls for the unwary. As a result, a proven process is required to ensure that you correctly identify and resolve performance
bottlenecks. …
• Identifying problematic SQL queries
• Analyzing a query execution plan
• Evaluating the effectiveness of the current indexes
• Avoiding bookmark lookups
• Evaluating the effectiveness of the current statistics
• Understanding parameter sniffing and fixing it when it breaks
• Analyzing and resolving fragmentation
• Optimizing execution plan caching
• Analyzing and avoiding statement recompilation
• Minimizing blocking and deadlocks
• Analyzing the effectiveness of cursor use
• Applying in-memory table storage and procedure execution
• Applying performance-tuning processes, tools, and optimization techniques to optimize SQL workloads”
https://www.safaribooksonline.com/library/view/sql-server-query/9781430267423/9781430267430_Ch01.xhtml
To see in depth review of most of these concepts https://msdn.microsoft.com/en-us/library/ff647793.aspx
DEMO: WORLDWIDEIMPORTERS
• World Wide Importers now replaces AdventureWorks sample database
https://blogs.technet.microsoft.com/dataplatforminsider/2016/06/09/wideworldimporters-the-new-sql-server-
sample-database/
• Entity Framework 2017 Community Edition is FREE!
• SQL Server Express 2016 / SQL Server Profiler is FREE
• Nuget packages: https://www.andrewcbancroft.com/2017/03/27/solving-spatial-types-and-functions-are-not-
available-with-entity-framework/
It Works… but not very efficient. Lets take a look at how much activity is generated using SQL Server profiler.
DYNAMIC PROXY – WE DON’T NEED ALL OF THIS EXTRA BAGGAGE. WE JUST WANT THE DATA.
DO WE REALLY NEED TO RUN ALL OF THESE QUERIES? WE AREN’T USING MOST OF THE DATA THAT IS BEING RETURNED
“the Database Engine Tuning Advisor can be a useful tool for analyzing the effectiveness of existing indexes and
recommending new indexes for a SQL workload. As the SQL workload changes over time, you can use this tool
to determine which existing indexes are no longer in use and which new indexes are required to
improve performance. It can be a good idea to run the wizard occasionally just to check that your
existing indexes really are the best fit for your current workload. This assumes you’re not capturing
metrics and evaluating them yourself. The Tuning Advisor also provides many useful reports for analyzing the SQL
workload and the effectiveness of its own recommendations. Just remember that the limitations of the tool prevent
it from spotting all tuning opportunities. Also remember that the suggestions provided by the DTA are only as good
as the input you provide to it. If your database is in bad shape, this tool can give you a quick leg up. If you’re
already monitoring and tuning your queries regularly, you may see no benefit from the recommendations of the
Database Engine Tuning Advisor.”https://www.safaribooksonline.com/library/view/sql-server-query/9781430267423/9781430267430_Ch10.xhtml
HOW CAN I BE PROACTIVE IN FINDING SLOW RUNNING PROCESSES? DATABASE ENGINE TUNING ADVISOR
DYNAMIC PROXY CLASS VS. POCO
By default, EF uses Change Tracking and uses an in-memory cache of all entities. (this is what makes those add, edit, and delete
operations so easy) Hmm…if only there were a way to say we weren’t going to need all of this overhead. Can we turn off
Change Tracking when we want to select data we know we won’t be editing?
We will get to that later.
To start, you may find transferring the data you need from the EF Context classes into POCO (or DTO) ones a common practice:
“The Entity Framework team recommends that you only use dynamic proxies if you find a need to; typically this would be around
performance tuning. POCO classes without proxies are usually simpler to interact with, as you don’t need to be aware of the additional
behaviors and nuances that are associated with proxies.”
https://www.safaribooksonline.com/library/view/programming-entity-framework/9781449317867/ch04.html
(In MVC, we deal mainly with POCO objects when we bind to the view. While it may be tempting to just bind your view to Entity
Framework classes – while it may work – aside from all the other design reasons why you shouldn’t like the presentation layer
shouldn't have any details about the data layer – just don’t.)
LAZY LOADING VS. EAGER LOADING
“8 Loading Related Entities
• The Entity Framework offers several different ways to load the entities that are related to
your target entity. For example, when you query for Products, there are different ways that
the related Orders will be loaded into the Object State Manager. From a performance
standpoint, the biggest question to consider when loading related entities will be whether
to use Lazy Loading or Eager Loading.
• When using Eager Loading, the related entities are loaded along with your target entity
set. You use an Include statement in your query to indicate which related
entities you want to bring in.
• When using Lazy Loading, your initial query only brings in the target entity set. But
whenever you access a navigation property, another query is issued against
the store to load the related entity.
• Once an entity has been loaded, any further queries for the entity will load it directly from
the Object State Manager, whether you are using lazy loading or eager loading.”
https://msdn.microsoft.com/en-us/data/hh949853.aspx
SHOULD I BE LAZY OR EAGER?
“I think it is good to categorize relations like this
• When to use eager loading
• In "one side" of one-to-many relations that you sure are used every where with main entity. like User property of an Article. Category
property of a Product.
• Generally When relations are not too much and eager loading will be good practice to reduce further queries on server.
When to use lazy loading
• Almost on every "collection side" of one-to-many relations. like Articles of User or Products of a Category
• You exactly know that you will not need a property instantly.
• Note: like Transcendent said there may be disposal problem with lazy loading.” -- farid bekran
https://stackoverflow.com/questions/31366236/lazy-loading-vs-eager-loading
.NET TALKING TO SQL SERVER IN SQL SERVERS NATIVE LANGUAGE…STRUCTURED QUERY LANGUAGE
▪ SQL Server has been around for a long time – don’t try to reinvent it!
▪ Managing and maintaining large SETS of data is what Microsoft has built SQL Server to
do.
▪ Most of the time performance problems around Entity Framework occur when querying
data stores.
▪ How do we go from an ORM or ENTITIES world and communicate when we want to
perform searches without losing all of the power Microsoft has built into SQL Server?
USING LINQ TO ENTITIES
From a LINQ to Entities query to a command tree…
“LINQ to Entities starts its journey in the LINQ APIs and is then passed to the Object Services API.
When you create a LINQ to Entities query, you are using syntax that is built into Visual Basic and
C# that has enhancements added by the Entity Framework. LINQ converts this query into a LINQ
expression tree, which deconstructs the query into its common operators and functions. The LINQ
expression tree is then passed to Object Services, which converts the expression tree to a
command tree.”
https://www.safaribooksonline.com/library/view/programming-entity-framework/9780596807276/ch10.html
HOW CAN WE IMPROVE? KEEPING THE RESULTS AS NARROW AS POSSIBLE – ONLY RETURN THOSE ROWS AND THOSE COLUMNS THAT ARE ABSOLUTELY NEEDED
“Return only the required data
Always return only the data that you need nothing more, nothing less. This approach reduces the data that we send across the wire (from the database server to the
web/application server).
For example, we would not use the following:
Select * from employees
Instead, we would use this:
Select FirstName, LastName from employees
The latter query would get only the required fields from the table, and, thus, only the required data is passed across to the calling client.”
https://www.safaribooksonline.com/library/view/learning-aspnet-core/9781786463838/ch11s05.html
“Using the correct column type and size for your database columns
When you want to use int as a datatype for a column, use an integer. Don't use double. This will save a lot of space if you have lots of rows in your
table.”
“ https://www.safaribooksonline.com/library/view/learning-aspnet-core/9781786463838/ch11s05.html
LINQ TO ENTITIES: PROJECTION TO POCO
Only selecting the fields we want in the output result! Only one trip to the database server. Lets look at this in
Profiler
STARTS WITH AND CONTAINS: NEWER IMPROVEMENT
“A key improvement to look for in the T-SQL generated by System.Data.SqlClient is smarter translation of queries
that use StartsWith or Contains. In .NET 3.5, Contains was not even supported. However, if you used
StartsWith in a query—for example, Contacts.Where(c=>c.LastName.StartsWith("T"))—the
database query that resulted performed poorly in the database. Now StartsWith and its newly
supported siblings, EndsWith and Contains, all result in queries that leverage T-SQL’s LIKE operator,
which takes advantage of indexing in SQL Server.
Note: The previous version of Entity Framework generated queries that forced the database to perform a full table
scan, which brought pain to the hearts of many database developers. The use of the LIKE operator in .NET 4 will be
a relief to many database professionals.”
https://www.safaribooksonline.com/library/view/programming-entity-framework/9780596807276/ch03.html
HOW CAN WE IMPROVE?
“Avoiding correlated subqueries
Correlated subqueries use values from their parent query, which in turn makes it run row by row.
This would significantly affect the query performance.”
http://www.sqlservice.se/sql-server-performance-death-by-correlated-subqueries/
TRICKING ENTITY FRAMEWORK INTO THINKING ITS LOOKING AT A TABLE
“In some cases you may want to simply map an entity to a view rather than a table. For
example, you may be mapping to a database that has a very large and confusing schema. To
simplify things, the database might contain a view that exposes the data for your entity with
more comprehensible column names. If the view is updatable, you can use the Entity
Framework to insert, update, and delete data as well as selecting it. Fortunately, most
databases, including SQL Server, use the same SQL syntax for interacting with views as they
do for tables. This means you can simply “lie” to Code First and tell it that the view is a
table. You do this by using the same configuration you use for naming tables.”
-- Programming Entity Framework: Code First, pg 154 | Chapter 7: Advanced Concepts
GIVING SQL SERVER A BIT OF WARNING
“Frequent Recompilation of Queries
One of the standard ways of ensuring a reusable execution plan, independent of values used in a
query, is to use a stored procedure or a parameterized query. Using a stored procedure to execute
a set of SQL queries allows SQL Server to create a parameterized execution plan.
A parameterized execution plan is independent of the parameter values supplied during the
execution of the stored procedure or parameterized query, and it is consequently highly reusable.
Frequent recompilation of queries increases pressure on the CPU and the query
execution time.”
https://www.safaribooksonline.com/library/view/sql-server-query/9781430267423/9781430267430_Ch01.xhtml
CREATING A VIEW…AND AN INDEX!
MAKE SURE YOU AREN’T ASKING SQL TO COMPARE APPLES TO ORANGES
• Look at the JOIN section of any query passed through SQL Server profiler and then look at the definition data types (and size
NOTE implicit conversion still takes place between char(10) and char(5)). Do the same in the where clause.
• Are you essentially asking JOIN on “5” = 5.000 ? While SQL Server will handle this for you it will need to perform an implicit
data conversion step somewhere along the query plan.
• NVARCHAR , VARCHAR, and CHAR are not the same so comparing them implicitly is very painful.
• Consider adding a computed persisted column that simply represents the data type of the column you want to join on, and
then index that. Persisted columns must be deterministic.
Pinal Dave has a great article on computed columns and performance here:
https://blog.sqlauthority.com/2010/08/03/sql-server-computed-column-persisted-and-performance/
If you ask Pinal Dave to connect with you on Linked In you will get information sent to you on a regular basis!
POCO ENTITIES AND CHANGE TRACKING PROXIES
• “The Entity Framework enables you to use custom data classes together with your data model without making any modifications to the data
classes themselves. This means that you can use "plain-old" CLR objects (POCO), such as existing domain objects, with your data model. These
POCO data classes (also known as persistence-ignorant objects), which are mapped to entities that are defined in a data model, support most of
the same query, insert, update, and delete behaviors as entity types that are generated by the Entity Data Model tools.
• Entity Framework can also create proxy classes derived from your POCO types, which are used when you want to enable features such as lazy
loading and automatic change tracking on POCO entities. Your POCO classes must meet certain requirements to allow Entity Framework to use
proxies, as described here: http://msdn.microsoft.com/library/dd468057.aspx.
• Chance tracking proxies will notify the object state manager each time any of the properties of your entities has its value changed, so Entity
Framework knows the actual state of your entities all the time. This is done by adding notification events to the body of the setter methods of your
properties, and having the object state manager processing such events. Note that creating a proxy entity will typically be more expensive than
creating a non-proxy POCO entity due to the added set of events created by Entity Framework.
• When a POCO entity does not have a change tracking proxy, changes are found by comparing the contents of your entities against a copy of a
previous saved state. This deep comparison will become a lengthy process when you have many entities in your context, or when your entities have
a very large amount of properties, even if none of them changed since the last comparison took place.
• In summary: you’ll pay a performance hit when creating the change tracking proxy, but change tracking will help you speed up
the change detection process when your entities have many properties or when you have many entities in your model. For
entities with a small number of properties where the amount of entities doesn’t grow too much, having change tracking proxies
may not be of much benefit.”
https://msdn.microsoft.com/en-us/data/hh949853.aspx
NOTRACKING QUERIES
If you are in a read-only scenario and want to avoid the overhead of loading the objects into the
ObjectStateManager, you can issue "No Tracking" queries. Change tracking can be disabled at the query level.”
Pros
• Improved performance over regular LINQ queries.
• Fully materialized objects.
• Simplest to write with syntax built into the programming language.
Cons
• Not suitable for CUD operations.
Certain technical restrictions, such as:
• Patterns using DefaultIfEmpty for OUTER JOIN queries result in more complex queries than simple OUTER JOIN
statements in Entity SQL.
• You still can’t use LIKE with general pattern matching.
Note that queries that project scalar properties are not tracked even if the NoTracking is not specified. For example:
var q = context.Products.Where(p => p.Category.CategoryName == "Beverages").Select(p => new {
p.ProductName });
This particular query doesn’t explicitly specify being NoTracking, but since it’s not materializing a type that’s
known to the object state manager then the materialized result is not tracked.”https://msdn.microsoft.com/en-us/data/hh949853.aspx
(If for some reason you are unable to use projection ex. You are told you must consume a certain factory method of a class when instantiating it…
you can use NoTracking. It’s a good practice to use NoTracking whenever you know you are wanting a “read only” recordset.)
NO TRACKING / NO IMPLICIT DATA CONVERSION / NARROW VIEW WITH INDEX
ALTERNATE METHODS AND A COMPARISON..
https://msdn.microsoft.com/en-us/data/hh949853.aspx
Stuck with using a stored procedure or some dynamic text? This page outlines not only
performance of different options but also provides sample programming!
6 Query Execution Options
ASK ALL THE QUESTIONS UP FRONT, INSTEAD OF SENDING THEM ONE AT A TIME
• Newer versions of Entity Framework allow you to map a stored procedure to return multiple
result sets.
• Asking for more than one result set up front will save time on the number of trips back and
forth between the database and your requesting code.
• Remember to not get too greedy with columns or rows – and try not to have a “universal
business adapter” solution to fix every problem. ie “lets have one stored procedure get us all
the data we will ever need across the application”
Khalid Abuhakmeh has a really good explanation of how to do this here:
http://www.khalidabuhakmeh.com/entity-framework-6-multiple-result-sets-with-stored-
procedures
REPOSITORY PATTERN ABUSE
• A repository pattern implementation can help you drastically improve development time…but remember you are adding yet another layer of abstraction.
• A sample of a generic repository has been included in the sample project.
https://docs.microsoft.com/en-us/aspnet/mvc/overview/older-versions/getting-started-with-ef-5-using-mvc-4/implementing-the-repository-and-unit-of-work-
patterns-in-an-asp-net-mvc-application
“The biggest performance mistake I see people make looks like:
var results = dbContext.ParentEntities.Where(x=> x.SomeCriteria == true).ToList();
// later down the call stack...
var results = repostitory.GetParents(criteria);
if (results.Count > 0)
// never actually use results, or do something like results.FirstOrDefault(), etc.
Then there are the sins of lazy loading or eager loading inappropriately, returning entire entity graphs where only a few fields are actually used.”
https://stackoverflow.com/questions/45065398/entity-framework-is-there-any-performance-improvement-on-querying-at-repository
ALTERNATIVE TO ENTITY FRAMEWORK AND THE REPOSITORY PATTERN
“I found that Entity Framework isn't always the best tool for the job. When it comes to complex schemas it's
sometimes nice to be able to construct your own SQL calls, or use stored procedures (if that's what floats your boat,
though in my opinion stored procedures should be used sparingly and for simple CRUD operations they are
generally unnecessary), this lead me to discover dapper-dot-net, a simple, efficient ORM that provides great
flexibility.”
https://www.brendanmckenzie.com/blog/2013/04/alternative-to-entity-framework-and-the-repository-
pattern
“It's popular lately to say that EF is a repository pattern and so implementing a repository pattern on top of it is
redundant.
https://www.reddit.com/r/dotnet/comments/3inh3z/do_i_need_the_repository_pattern_if_i_use_entity/
QUESTIONS?
Connect with me on linked in!
https://www.linkedin.com/in/paul-nelson-098596