FindPagesWithCriteria and Performance

by: Steve Celius

Lately I have heard from several developers that FindPagesWithCriteria is evil, and should be avoided at all costs. I’ve heard it from different people, in different unrelated cases, and some comments are needed.

Yes, FindPagesWithCriteria can slow down performance on your site. I emphasize “can” because in most cases, the FindPagesWithCriteria is not the reason your site is slow.

The reason is the amount of data you get back from FindPagesWithCriteria!

I recently helped optimize a template that took 17 seconds to render. The FindPagesWithCriteria call took 500ms. The consensus was that FindPagesWithCriteria is bad and needed to go. A run with DotTrace showed us what was really going on. Heavy (and unnecessary) data manipulation on way to many PageData objects.

The code was churning through ~49.000 PageData objects, making writeable clones, and displayed about 50 pages in a list (some times more, mostly less.)

Perhaps the developer had been reading about this in the SDK:

“Performance tip! Try to setup as few criterias as possible for your selection because every criteria will resolve into a database lookup.”

Sure, isolated, this is true (on a good day), but in most cases, getting too much data will hurt performance a lot more than adding more criterias.

The case I mentioned above was rewritten with more criterias, which returned only the pages the list needed, avoiding post filtering and we also stopped making writable clones (which is something you should seldom have to do in lists anyway.) The FindPagesWithCriteria takes less than 100ms now, and rest of the rendering is not worth measuring. It is blazing fast.

Here are some FindPagesWithCriteria guidelines:

  1. Beware of large results – a big PageDataCollection requires all PageData objects to be fetched from the database and cached.
  2. Add as many criterias you need to get only the data you need. Filtering in the database is always faster than in your code
  3. Think ahead. The search you do today might behave quite different a year from now, when you have more data.
  4. If you need many FindPagesWithCriteria calls in your solution, consider caching (see the cache framework for more information).
  5. If you need “the last <n> news articles” type of functionality, consider calling FindPagesWithCriteria many times with increasing time span instead of getting all news and then sort + filter.
  6. Beware of sorting and filtering. Do not add 1000 pages to a NewsList and then set the MaxCount. It will sort all pages and the limit the count in memory. Complete waste of resources.
  7. Remember to filter the collection for visitors if you’re presenting it outside a list that does that for you. Do not filter if the list does it for you.

FindPagesWithCriteria is not evil. It is very useful and most sites need it. Just beware of what it returns.

23 August 2009


Comments

  1. Really nice article and definately worth bringing up. FindPagesWithCriteria has gotten a little to much "bad" credit lately. I think it is a extremely useful function and as you describe; used in the right way won't probably be the Achilles' heel in the application.
  2. It seems so obvious now :) These guidelines should be added to the SDK as well.
  3. Great post. One more thing to remember, though: When using the SearchDataSource, avoid combining free-text queries with criterias (e.g. Find all pages with the word "slow" in the last 14 days). This will cause it to do the free-text search, load all pagedata objects from it and then use the criteria for in-memory filtering.
  4. I'm glad Steve is back on the bloggwagon!
  5. I think EPiServer 4 is the reason for FindPagesWthCriteria has a bad repution. With the PageData cache scheme in EPiServer 5 it have improved a lot. But if you take a look at the code it still populate one and one page with one database call (if its not in cache). So the first time you use it it will definitift use more than 500ms :)
  6. Thank you all for your comments. Anders; this is exactly why you need to make sure the result set is not larger than it needs to be. Less pages, less GetPage() calls. Easier on the memory consumption too.
  7. Thats the quick fix Steve. The more robust approced should be that instead of featch one and one page, the API should fetch all the data an populate it in one go. So instead of n dataaccess you could do it in 1. Have done something like this in EPiServer 4. And its doable.
  8. I think this is the same problem I've seen with dynamic properties, used wrongly it can have performance impacts (perhaps more so in epi 4 than 5, I wasn't around in the epi world then so not sure). New developers often wants some sort of "best practice" and perhaps hear that FindPagesWithCriteria is slow and just to be safe decide to not use it. But as this post shows, it's not evil in itself you just have to be fairly certian of what you're doing and have a general idea of the inner workings of the method. The newcomer probably won't be in that position which is why information such as this would be a great assett to the sdk.
  9. Anders, getting just the data you need, and not more, is never the quick fix, it is the correct thing to do. :-) On the other side, making the API do what you say would of course help a lot (though memory is also a limited resource.) Instead of getting everything from the database, I'd rather see sortorder, startindex and maxcount parameters on the FPWC call to allow the database to do the sorting, and allowing paging too. (Security filtering in this case is a hazzle though.)
  10. Steve : lol, so we can agree that the correct thing would be that FindPagesWithCriteria should be able to sort and filter based on count and premisions.
  11. Thanx for a great post! Has the _order_ of the criterias you put in the criteria-collection for the FindPagesWithCriteria any impact on performance? And does searching on the built-in properties give a better performance on FindPagesWithCriteria?
  12. Jonatan, I haven't looked into the order or built-in vs. custom properties regarding performance. In most cases, that is wasted, as the actual FindPagesWithCriteria call is not the one that takes time. Add as many criterias (built-in or custom) as you'd like, as long as it helps you find only the content you need, and not more.
  13. Found this blog about speed: http://thisisnothing.wordpress.com/2009/08/27/a-quick-comparsion-between-findpageswithcriteria-and-a-recursive-getchildren/ a
Post a comment    
User verification Image for user verification  
Steve Celius

About me

I work for EPiServer in Norway, mostly with technical stuff. Trying to keep up with all the new stuff from the development team. I also hang out on the EPiCode project, why don't you come join us?

You can also follow me on Twitter:

 

Number of visits to this page:
1224

Top five pages on the site:

 

Syndications


Archive


Tag cloud

EPiTrace logger