Home > Ask the Systems Channel Experts > Database Administration Services Questions & Answers > Architectural solutions to Web-based applications with large datasets for VARs
Ask The Systems Channel Expert: Questions & Answers
EMAIL THIS

Architectural solutions to Web-based applications with large datasets for VARs

Retired Expert - Hilary Cotter EXPERT RESPONSE FROM: Retired Expert - Hilary Cotter

Pose a Question
Other Systems Channel Categories
Meet all Systems Channel Experts
Become an Expert for this site


Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us   


>
QUESTION POSED ON: 27 February 2007
What is the best way to configure a database to support web-based applications with large dynamic datasets?

>
EXPERT RESPONSE
This is always a challenge. Throwing hardware at it rarely solves the problem. The best approach is an architectural one. These are some questions VARs should consider:
  • Can caching be used? File server and Web servers are optimized for sending large files, whereas RDBMs are optimized for set based operations and do not always scale well for large operations. Caching will work best if the dynamic queries repeat themselves or if portions of them are updated. For example, most search engines take advantages of the fact that 90% of the searches are on the same topics, and these search results are cached and sent to the client without hitting the full-text catalogs at all. These cached results are updated continuously. Naturally this will only work if you know in advance what the dynamic data sets will look like.
  • Can the rest be batched and send to the web client asynchronously? Programming for Web clients require special programming techniques in dealing with large dynamic datasets to prevent the request from timing out. On the database you want the queries to run as fast as possible. Time spent in denormalization and creating covering indexes will provide the greatest impact on performance.
  • Can you use paging? Paging will only display the first 10, 25, or 100 results on the client. If the Web application does not need to display or consume the entire dynamic data set at one time you can return the first 100 results to the Web client and then generate the entire dynamic data set in the background so that repeat request will be returned from this pre-generated dynamic data set.
  • Can you use query notifications? Query notifications will keep a dynamic dataset updated with newly inserted rows. It is a feature of SQL Server 2005 and ADO.Net 2.0. If a change occurs in the data underlying the dynamic dataset a notification will be made to refresh the dataset.
  • Is the RDBMs the right tool for the task? I attended a presentation where a company displayed their reporting tool which displayed all sort of aggregated content. The backend was a SQL Server database. As the reporting tool dealt exclusively with aggregations and drill downs the natural choice should have been Analysis Services.


Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us   


RELATED CONTENT
Database Administration Services
What opportunities are there for VARs in a SQL Server upgrade?
How to decide which disaster recovery strategy is right for customers?
Why outsource database management services to a VAR?
Offer SQL Server security services
Integrating Microsoft Office with SQL Server
What should I expect when migrating custom applications from SQL Server to MySQL?
SQL Server database security
SQL Server capacity planning
SQL Server 2005 business suite and Microsoft Office integration
Controlling Microsoft SQL Server sprawl

Data Warehousing
Web-based database support
DBA services can make VARs money
Service-oriented management in the channel
Business intelligence as a managed service
Database reliabilty during the holiday season
Oracle database management client concerns
Becoming a remote DBA services provider
Plan your remote DBA services
Data integration is a process
Supporting real-time BI with real-time ETL

Database Availability and Scalability
High-availability options in SQL Server 2008
Three low-cost approaches to high-availability databases
What are the best SMB databases?
What skill sets do I need to support a fault-tolerant, high-availability system?
How can I help clients meet service-level requirements?
Oracle, system integrator develop new HR product
Providing database services: The initial client meeting
Controlling Microsoft SQL Server sprawl
Database mirroring best practices for VARs
Web-based database support

RELATED RESOURCES
2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems
Search Bitpipe.com for the latest white papers and business webcasts
Whatis.com, the online computer dictionary



Search and Browse the Expert Answer Center
Search and browse more than 25,000 question and answer pairs from more than 250 TechTarget industry experts.
Browse our Expert Advice

HomeTopicsITKnowledge ExchangeTipsMultimediaWhite PapersBlogsEvents
About Us  |  Contact Us  |  For Advertisers  |  For Business Partners  |  Site Index  |  RSS
SEARCH 
TechTarget provides enterprise IT professionals with the information they need to perform their jobs - from developing strategy, to making cost-effective IT purchase decisions and managing their organizations' IT projects - with its network of technology-specific Web sites, events and magazines.

TechTarget Corporate Web Site  |  Media Kits  |  Reprints  |  Site Map




All Rights Reserved, Copyright 2006 - 2008, TechTarget | Read our Privacy Policy 		  TechTarget - The IT Media ROI Experts