The MegaStore Application

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)


Databases and Web-Based Applications. The MegaStore Application Advanced Internet-based Electronic Commerce Technologies for Music Industry. Ammar Benabdelkader University of Amsterdam. December 1, 2003 UvA, Amsterdam. Motivation. - PowerPoint PPT Presentation

Transcript of The MegaStore Application

No Slide TitleDatabases and Web-Based Applications
Data classification and cataloguing are required
Related information needs to be properly inter-linked
Efficient storage, and access of large multimedia data is supported
Short response time for on-line requests must be guaranteed
High data transfer rates must be provided
Furthermore, public information must be separated from the private information that need to be securely kept at the place where it belongs.
A. Benabdelkader ©UvA, 2002
An efficient architecture must be designed addressing the need to provide the user of electronic commerce with an environment through which he can experience as sufficiently close to real life shopping environment. The approach considers the following aspects:
Security for data access and users authentication
Suitable user friendly interfaces
Private data need to be protected
Public data is made available to the Internet users
The MegaStore system aims at the design and set-up of the necessary database structure and platform architecture for advanced e-commerce applications
A. Benabdelkader ©UvA, 2002
From the analysis of music industry application, the following aspects are identified:
The data is geographically distributed over the network
Information about music is classified into two main categories:
The general information stored at the directory services (database catalogue) and accessible by all the Internet users
The raw music data that can only be accessible by the music storekeepers at music centers or burning towers
Depending on the user profile and authorization, only a part of the information can be accessed and users need not to know about the data distribution
The real music data must be securely transferred through a dedicated network
High bandwidth connection is necessary to handle raw music data that need to be transferred between the music storage centers and the burning towers
Low latency network connection is necessary to support the huge number of users expected to connect to the system
Problem Analysis
The designed system architecture involves the following components:
The back-end system, including the database engine and the predefined networking connection between the MegaStore system components
The front-end system, including:
The Internet–Shop interface, where a user from home (or work place) can search for music, listen/watch to the audio/video clips, and order CDs, and
The Shop-in-a-Shop interface, where the music storekeeper can fetch on-line the real music data from its original source in order to burn at run-time the requested music CDs
A. Benabdelkader ©UvA, 2002
Parallel Distributed
Database Server
Song Artists
Album Entry
Song Entry
User Login
Level 3
Artist Entry
Level 2
Artist Songs
Show Help
Show Songs
Custom Orders
Show Artists
Order albums
Album Songs
Show Albums
System Implementation
The database schema is implemented on top of the Matisse object-oriented database system
The Matisse back-end database runs on a cluster composed of 20 nodes.
The MegaStore Internet interface is implemented using an NT front-end machine, that is in turn connected to the underlying back-end database.
The Internet-shop server prototype is implemented using a combination of the following software technologies:
JavaScript and Vbscript for tips programming,
Active Database Objects (ADO) for database access, and
HTML for text formatting
the server implementation is made possible using the Active Server Pages (ASP) environment that allows the combination of different software technologies in one single environment.
A. Benabdelkader ©UvA, 2002
ODL is used to support the portability of database schemas across conforming ODBMSs.
OIF is used to exchange objects between databases and provide database documentation.
Universal data access through standards and middle ware solutions (XML, ODBC, JDBC, JAVA, etc.).
F- B
Legacy systems
Management of Large Multi-Media Data
An efficient architecture must be designed for manipulation of large data sets. Therefore, Data management mechanisms must be addressed in such a way that:
Data needs to be properly searched, retrieved, published, inter-linked, and compared to other data,
Information security is preserved,
Performance issues are improved,
Private data is protected, while published data is made available to the outside users.
A. Benabdelkader ©UvA, 2002
External Data Link Approach
One-Database Storage Approach
Parallel/Distributed Database Server
Comparing related data in different applications,
Supporting the ability to query and modify the data using appropriate query languages,
Searching the stored information and supporting efficient access to data items,
Preserving the system coherency, data is scattered in various files of different formats.
Applications are based on direct access to local file system: they are hard to maintain and to extend.
1- File System Approach
A database catalogue is used together with the file system:
Database provides references to all objects stored locally or remotely
This approach solves problems related to database overload and improves DB performance
Data is distributed so that it is physically located closest to intensive usage sites
Result files can be archived at (or close to) the point where they are generated
Proper data distribution reduces access bottlenecks at individual sites
2- External Data Link Approach
Database catalogue consistency: referenced binary objects can be updated/removed without notifying the database catalogue maintainer
Solution: a specific module that automatically and periodically checks the availability of the referenced objects against the database catalogue
Security issue: referenced objects, which are usually stored in a public location, are not secure.
Solution: development of a remote file server, through which, the file access and user authentication are controlled based on the database catalogue information
2- External Data Link Approach (Cont.)
A. Benabdelkader ©UvA, 2002
In ideal case, if a database is being created to manage the meta-data, then that database must also store the large scientific data
This approach solves the problem of keeping the meta-data synchronized with the archive
However, since the scientific binary data is of huge size:
It is very costly to store/access the data as large binary objects within the database itself
Access mechanisms to binary objects require extra encoding/decoding facilities
3- One-Database Storage Solution
Database Server
This approach uses a database repository to store the general information, and a distributed database server to store large objects:
The database repository is better exploited for cataloguing, indexing, and searching facilities
The database server enforces the issues related to security for access, concurrency control, and information visibility rights.
A. Benabdelkader ©UvA, 2002
The parallel/distributed database framework provides the MegaStore web server with efficient access to the raw music data. The nodes (music stores) of the distributed MegaStore server are inter-connected, making it possible for specific users to connect to any node in the distributed server and to request an object, without the need to know where that object actually resides
The distributed database supports the following required functionalities:
Provides a way for managing huge amount of data
Data is securely kept at geographically distributed music centers
Data is stored only at the point(s) where it belongs
Data is visible from any node (music center) within the cooperation community
Data is efficiently transferred between the nodes in short response time
Distributed Parallel Server Extension
A. Benabdelkader ©UvA, 2002
Design and development of an e-MegaStore application that seamlessly fits several emerging applications and supports their model of operations
Design Methodology:
Value Adding Partners
Content Suppliers Partners
A. Benabdelkader ©UvA, 2002
Short Clips
A. Benabdelkader ©UvA, 2002
A. Benabdelkader ©UvA, 2002
A. Benabdelkader ©UvA, 2002
A. Benabdelkader ©UvA, 2002
A. Benabdelkader ©UvA, 2002
The main idea behind the developed framework is to design a comprehensive system to support advanced web-applications with two specific characteristics:
to hold large data sets and
to manage multimedia information
Thus, the MegaStore system can be considered as a general implementation approach that proves the validity of the proposed architecture and design.
From this framework, other applications in biology, medicine, and system engineering that share the same characteristics can benefit.
A. Benabdelkader ©UvA, 2002
Matisse DBMS:
A. Benabdelkader, H. Afsarmanesh, L. O. Hertzberger. MegaStore: Advanced Internet-based Electronic Commerce Service for Music Industry. In proceedings of 11th IEEE International Conference on Database and Expert Systems Applications - DEXA'2000, Pages 869-878, London - Greenwich, United Kingdom, 2000.
A. Benabdelkader, H. Afsarmanesh, L. O. Hertzberger. The Virtual MegaStore System Implementation. Technical Report CS-99-05, Faculty of Science, Research Institute Computer Science, University of Amsterdam, 1999.
A. Benabdelkader, H. Afsarmanesh, L. O. Hertzberger. The Virtual MegaStore System Architecture: Analysis and Design. Technical Report CS-99-04, Faculty of Science, Research Institute Computer Science, University of Amsterdam, 1999.