This project is read-only.

Getting started

Concepts

RaptorDB is a json based key value store with map functions which map documents to views for querying. So at its core you save a document entity with Save() and Fetch() the same.

A key value store is very useful, but for most applications you need to query and aggregate parts of documents, so you need to define views to extract information from those documents. Extracting information like this is done for performance reasons.

You can only have 1 instance of RaptorDB accessing the data folder (while files are opened in shared mode so you can take hot backups, two instances cannot share the same files and folders).

While the noSql movement advocates "schema-less" designs, this does not mean the absence of a schema, but less dependency on a schema or schema isolation i.e. the ability to change things without breaking other parts, because to do anything with performance you will need to know your data types and structures otherwise queries would be a limited full text search (limited in the way that text data loses type information).

Installing

You can use RaptorDB in your projects by installing via nuget and searching for RaptorDB_doc or by manually adding 2 DLL references :
  • RaptorDB.dll : the main server code
  • RaptorDB.Common.dll : common library for both embedded and client/server

Your first Project

The complete source code : Cannot resolve file macro, invalid file name or id.

For demonstration purposes we will create a console application project (you can create any kind of app with Visual Studio). Our first project will use RaptorDB in embedded mode, meaning that the database will share the memory of the console process.

Since RaptorDB needs to keep a lot of state then we should create a global variable for it so we can access it from throughout our code and have 1 instance:
    class Program
    {
        static RaptorDB.RaptorDB rdb; // 1 instance

        static void Main(string[] args)
        {
            rdb = RaptorDB.RaptorDB.Open("data"); // a "data" folder beside the executable
            RaptorDB.Global.RequirePrimaryView = false;

            DoWork();

            Console.WriteLine("press any key...");
            Console.ReadKey();
            Console.WriteLine("\r\nShutting down...");
            rdb.Shutdown(); // explicit shutdown
        }
    ...
Note that we are explicitly shutting down so RaptorDB can cleanly save data to disk (if you don't do this you might notice RaptorDB rebuilding the views as it didn't shutdown cleanly).

For the purpose of our first example we will disable the need to have primary views defined for our documents and essentially use RaptorDB as a key value store.

Now lets save some data:
        static void DoWork()
        {
            Console.Write("Inserting 100,000 documents...");
            int count = 100000;

            for (int i = 0; i < count; i++)
            {                
                var inv = CreateInvoice(i);

                // save here
                rdb.Save(inv.ID, inv);
            }

            Console.WriteLine("done.");
        }
From the above code you can see that every document (SalesInvoice) has a Guid (ID) associated with it and we save that with the invoice and this is done with the Save() method which takes the ID and the document as parameters. For more user friendly data we are using Faker to generate the data in the CreateInvoice() method :
        static SalesInvoice CreateInvoice(int counter)
        {
            // new invoice
            var inv = new SalesInvoice()
            {
                Date = Faker.DateTimeFaker.BirthDay(),
                Serial = counter % 10000,
                CustomerName = Faker.NameFaker.Name(),
                NoCase = "Me " + counter % 10,
                Status = (byte)(counter % 4),
                Address = Faker.LocationFaker.Street(),
                Approved = counter % 100 == 0 ? true : false
            };
            // new line items
            inv.Items = new List<LineItem>();
            for (int k = 0; k < 5; k++)
                inv.Items.Add(new LineItem() { Product = "prod " + k, Discount = 0, Price = 10 + k, QTY = 1 + k });

            return inv;
        }
If we know a Guid then we can get the document with :
var obj = rdb.Fetch(known_guid); // obj will be a SalesInvoice from the above data
Now obviously we want to be able to query with more than just a Guid that we know so we need to define a view on our SalesInvoice type, so our view is something like (we will skip what the code means for now):
    public class SalesInvoiceViewRowSchema : RDBSchema
    {
        public string CustomerName;
        public string NoCase;
        public DateTime Date;
        public string Address;
        public int Serial;
    }

    [RegisterView]
    public class SalesInvoiceView : View<SalesInvoice>
    {
        public SalesInvoiceView()
        {
            this.Name = "SalesInvoice";
            this.Description = "A primary view for SalesInvoices";
            this.isPrimaryList = true;
            this.isActive = true;
            this.BackgroundIndexing = true;
            this.Version = 1;

            this.Schema = typeof(SalesInvoiceViewRowSchema);

            this.Mapper = (api, docid, doc) =>
            {
                api.EmitObject(docid, doc);
            };
        }
    }
and we register the view with RaptorDB on start-up :
        static void Main(string[] args)
        {
            rdb = RaptorDB.RaptorDB.Open("data"); // a "data" folder beside the executable
            RaptorDB.Global.RequirePrimaryView = false;

            Console.WriteLine("Registering views..");
            rdb.RegisterView(new SalesInvoiceView());
        ...
our DoWork() becomes :
        static void DoWork()
        {
            long c = rdb.DocumentCount();
            if (c > 0) // not the first time running
            {
                var result = rdb.Query<SalesInvoiceViewRowSchema>(x => x.Serial < 100);
                // show the rows
                Console.WriteLine(fastJSON.JSON.ToNiceJSON(result.Rows, new fastJSON.JSONParameters { UseExtensions = false, UseFastGuid = false }));
                // show the count
                Console.WriteLine("Query result count = " + result.Count);
                return;
            }

            Console.Write("Inserting 100,000 documents...");
            int count = 100000;

            for (int i = 0; i < count; i++)
            {                
                var inv = CreateInvoice(i);

                // save here
                rdb.Save(inv.ID, inv);
            }

            Console.WriteLine("done.");
        }
Some of things to notice :
  • we are using the beautified output of fastJSON to show the data return which is built-in to RaptorDB.
  • on re-running our changed code, the view was automatically rebuilt with the data we already saved, and our query works (you will have to rerun a second time for the query to show results since the engine will be rebuilding the view in the background and will not be available at the time the query is executed).
  • we did nothing to define indexes and all that was taken care of for us.
  • the data was returned to us in a typed format much like a typed DataSet if you are familiar with that.
  • data was returned from the view we defined and not the original documents
  • the entire process is type safe and the compiler will help you if a mistake was made, rather than at runtime much like string SQL queries (if you need dynamic queries you can use string filters also, but obviously the compiler can't help you with mistakes).

Now Query() has a few overloads to control paging and sorting based on a criteria :
    // string based view names and object row returns
    public Result<object> Query(string viewname);
    public Result<object> Query(string viewname, string filter);
    public Result<object> Query(string viewname, int start, int count);
    public Result<object> Query(string viewname, string filter, int start, int count);
    public Result<object> Query(string viewname, string filter, int start, int count, string orderby);

    // string based filters and typed row returns (the view is determined by the schema you used)
    public Result<TRowSchema> Query<TRowSchema>(string filter);
    public Result<TRowSchema> Query<TRowSchema>(string filter, int start, int count);
    public Result<TRowSchema> Query<TRowSchema>(string filter, int start, int count, string orderby);

    // LINQ predicate filters and typed row returns (the view is determined by the schema you used)
    public Result<TRowSchema> Query<TRowSchema>(Expression<Predicate<TRowSchema>> filter);
    public Result<TRowSchema> Query<TRowSchema>(Expression<Predicate<TRowSchema>> filter, int start, int count);
    public Result<TRowSchema> Query<TRowSchema>(Expression<Predicate<TRowSchema>> filter, int start, int count, string orderby);

Oooops! things change

Now say we have underestimated what we need in our queries and much like life, requirements change, so what do we do?

Well! for RaptorDB it is simple, all we need to do is change our schema and view like below :
    public class SalesInvoiceViewRowSchema : RDBSchema
    {
        public string CustomerName;
        public string NoCase;
        public DateTime Date;
        public string Address;
        public int Serial;
        public byte Status;    // added to the view 
        public bool? Approved; // added to the view
    }

    [RegisterView]
    public class SalesInvoiceView : View<SalesInvoice>
    {
        public SalesInvoiceView()
        {
            this.Name = "SalesInvoice";
            this.Description = "A primary view for SalesInvoices";
            this.isPrimaryList = true;
            this.isActive = true;
            this.BackgroundIndexing = true;
            this.Version = 2; // <- increment when you make changes and want a rebuild 

            this.Schema = typeof(SalesInvoiceViewRowSchema);

            this.Mapper = (api, docid, doc) =>
            {
                if (doc.Status == 0)  // status = 0 means a draft and should not be saved 
                    return;
 
                api.EmitObject(docid, doc);
            };
        }
    }
The key is the Version property which tells RaptorDB to rebuild the view if it is older. On restarting our application the view is rebuilt and our queries reflect the changes.

Side Note

  • The above view is primary list and we probably don't want to not save since there will be no reference to retrieve the SalesInvoice from, you would probably do this kind of filtering based on Status for example in other detail or summation views. The above is only for demonstration purposes.
  • As you can see we can change the view schema and/or how documents are processed and all we need to do is change the Version number to let RaptorDB know when to rebuild the view's contents.

Full source

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

using RaptorDB;
using RaptorDB.Common;

namespace rdbtest
{

    #region [  entities  ]
    public class LineItem
    {
        public decimal QTY { get; set; }
        public string Product { get; set; }
        public decimal Price { get; set; }
        public decimal Discount { get; set; }
    }

    public class SalesInvoice
    {
        public SalesInvoice()
        {
            ID = Guid.NewGuid();
        }

        public Guid ID { get; set; }
        public string CustomerName { get; set; }
        public string NoCase { get; set; }
        public string Address { get; set; }
        public List<LineItem> Items { get; set; }
        public DateTime Date { get; set; }
        public int Serial { get; set; }
        public byte Status { get; set; }
        public bool Approved { get; set; }
    }
    #endregion

    #region [  view definition  ]
    public class SalesInvoiceViewRowSchema : RDBSchema
    {
        public string CustomerName;
        public string NoCase;
        public DateTime Date;
        public string Address;
        public int Serial;
        public byte Status;
        public bool? Approved;
    }

    [RegisterView]
    public class SalesInvoiceView : View<SalesInvoice>
    {
        public SalesInvoiceView()
        {
            this.Name = "SalesInvoice";
            this.Description = "A primary view for SalesInvoices";
            this.isPrimaryList = true;
            this.isActive = true;
            this.BackgroundIndexing = true;
            this.Version = 1;

            this.Schema = typeof(SalesInvoiceViewRowSchema);

            this.Mapper = (api, docid, doc) =>
            {
                //if (doc.Status == 0)
                //    return;

                api.EmitObject(docid, doc);
            };
        }
    }
    #endregion

    class Program
    {
        static RaptorDB.RaptorDB rdb; // 1 instance

        static void Main(string[] args)
        {
            rdb = RaptorDB.RaptorDB.Open("data"); // a "data" folder beside the executable
            RaptorDB.Global.RequirePrimaryView = false;

            Console.WriteLine("Registering views..");
            rdb.RegisterView(new SalesInvoiceView());

            DoWork();

            Console.WriteLine("press any key...");
            Console.ReadKey();
            Console.WriteLine("\r\nShutting down...");
            rdb.Shutdown(); // explicit shutdown
        }

        static void DoWork()
        {
            long c = rdb.DocumentCount();
            if (c > 0) // not the first time running
            {
                var result = rdb.Query<SalesInvoiceViewRowSchema>(x => x.Serial < 100);
                // show the rows
                Console.WriteLine(fastJSON.JSON.ToNiceJSON(result.Rows, new fastJSON.JSONParameters { UseExtensions = false, UseFastGuid = false }));
                // show the count
                Console.WriteLine("Query result count = " + result.Count);
                return;
            }

            Console.Write("Inserting 100,000 documents...");
            int count = 100000;

            for (int i = 0; i < count; i++)
            {                
                var inv = CreateInvoice(i);

                // save here
                rdb.Save(inv.ID, inv);
            }

            Console.WriteLine("done.");
        }

        static SalesInvoice CreateInvoice(int counter)
        {
            // new invoice
            var inv = new SalesInvoice()
            {
                Date = Faker.DateTimeFaker.BirthDay(),
                Serial = counter % 10000,
                CustomerName = Faker.NameFaker.Name(),
                NoCase = "Me " + counter % 10,
                Status = (byte)(counter % 4),
                Address = Faker.LocationFaker.Street(),
                Approved = counter % 100 == 0 ? true : false
            };
            // new line items
            inv.Items = new List<LineItem>();
            for (int k = 0; k < 5; k++)
                inv.Items.Add(new LineItem() { Product = "prod " + k, Discount = 0, Price = 10 + k, QTY = 1 + k });

            return inv;
        }
    }
}

Last edited Mar 17, 2015 at 7:13 AM by MGholam, version 14

Comments

No comments yet.