Storing an object which has a byte[] field?

Jan 23, 2014 at 2:01 AM
Edited Jan 23, 2014 at 2:03 AM
When registering a view via
rap.RegisterView(new myRaptorDbEntityView());

getting
An exception of type 'System.ArgumentException' occurred in mscorlib.dll but was not handled in user code

Additional information: GenericArguments[0], 'System.Byte[]', on 'RaptorDB.TypeIndexes`1[T]' violates the constraint of type 'T'.
The classes are defined as
public class myRaptorDbEntityView : View<myRaptorDbEntity>
{
    public myRaptorDbEntityView()
    {
        Name = "myRaptorDbEntityView";
        Description = "A primary view for myRaptorDbEntity";
        isPrimaryList = true;
        isActive = true;
        BackgroundIndexing = true;
        Schema = typeof (mySchemaForRaptorDb);
        AddFireOnTypes(typeof(myRaptorDbEntity));
        Mapper = (api, docid, doc) => api.Emit(docid, doc);
    }
}

public class mySchemaForRaptorDb : RDBSchema // define the schema for this view
{
    public byte[] EncShortMsg;
}

public class myRaptorDbEntity
{
    public myRaptorDbEntity()
    {
        ID = Guid.NewGuid();
    }
    
    public Guid ID { get; set; }

    public byte[] EncShortMsg { get; set; }
}
And the IO/usage is
// WRITING
var item = new myRaptorDbEntity();
rap.Save(item.ID, item.EncShortMsg);

// READING (Every item)
var q = rap.Query<myRaptorDbEntity>(s => true);
var qr = q.Rows;
if (qr != null)
{
    foreach (var item in qr)
    {
        // do work with item.EncShortMsg ...
    }
}
Any idea what might be wrong? Pulling my hair over here :( ...

EDIT: This is the simplest case ... other entities we're saving have a more complex mix of various data types (like int, float, byte, byte[], string etc in a single class/object).

I have to admit, for a schema-less design, there is much fighting with the view and schema...
Jan 23, 2014 at 6:54 AM
Thanks Sid!

Currently byte[] is not supported for columns in views, so that is why you are getting the error.

You can try and save it as a string in the meantime.

Also in your save you should be doing :
var item = new myRaptorDbEntity();
rap.Save(item.ID, item);  // the full object
Jan 23, 2014 at 6:17 PM
Ok, it now works but I've added a speedbump in our db IO by forcing base64 conversion on every read and every write.
Jan 23, 2014 at 6:46 PM
You should think of views as things you want to query and show in UI's, so you would use int,decimal,string etc. and not byte[] which you can't really search.
Jan 23, 2014 at 7:22 PM
Edited Jan 23, 2014 at 7:27 PM
Also, I had to modify the classes with 4 fixes, two of which you mentioned. They are marked below as "Fix: #". There also appear to be two very weird issues, marked below as "Issue #". The MOST confusing is the fact that the RaptorDb API appears inconsistent ... I save the business logic object (say, TValue) but I read back an object of some artificially construed "View" (say TView) ?? This itself might warrant a separate post ... but in the most simple case, it would be like saving a float by reading back a string (and then forcing app logic to regenerate the float)

For a NoSQL object/document or K-V store, if I give you objects of one type (TValue), when I query it back I expect it back in the same type (TValue). I can understand the desire for views but I shouldn't be forced into using views where they make no sense (eg: our code has NO UI elements - at all). A generalized db engine should be generalized instead of forcing users into arbitrary types or schemas. At the very least (following "convention over configuration") the default view should be that of the original type itself, with special configuration to override that when the application desires to introduce a mis-match between read and write types. IMHO, forcing views and schemas down a user's throat adds a lot more friction to development especially when developers expect no schema's in a NoSQL design.

All of the above can be distilled into an IRaptorDb interface that should have
// Missing: Simple API, get back same type as original store
Result<T> Query<T>(Expression<Predicate<T>> filter);
// Current (conceptually): Get back after transforming to a view type
Result<TView> QueryGetView<TView>(Expression<Predicate<TView>> filter);
where T can be some basic data types (byte[] should be a basic data type). For all other data types, you could either serialize to JSON or let the user serialize it by themselves (via Protocol Buffers/Thrift/Avro/etc) to string or byte[].

Am I missing something fundamental here? Don't get me wrong, I'm very happy you're doing this and I want RaptorDb to succeed. It's just very weird (and unproductive) to have that sort of an interface into a db engine.

PS: As a sidenote, your IRaptorDb interface is also missing RegisterView forcing users to spawn a specific type of RaptorDb.RaptorDb instead of type IRaptorDb (which works against any clean interfaces design or dependency injection)

Code below:
// View
public class myRaptorDbEntityView : View<myRaptorDbEntity>
{
    public myRaptorDbEntityView()
    {
        Name = "myRaptorDbEntityView";
        Description = "A primary view for myRaptorDbEntity";
        isPrimaryList = true;
        isActive = true;
        BackgroundIndexing = true;
        Schema = typeof(mySchemaForRaptorDb);
        AddFireOnTypes(typeof(myRaptorDbEntity));
        
        // Fix #1: Emit order should fit into RaptorDb schema object
        Mapper = (api, docid, doc) => api.Emit(docid, doc.EncShortMsg);
    }
}

// Schema
public class mySchemaForRaptorDb : RDBSchema 
{
    // Fix #2: Disambiguated field name via 'Schema' prefix
    public byte[] SchemaEncShortMsg; 
}

// Business logic entity
public class myRaptorDbEntity
{
    public myRaptorDbEntity()
    {
        ID = Guid.NewGuid();
    }
    
    public Guid ID { get; set; }

    public byte[] EncShortMsg { get; set; }
}
And the IO/usage is
// WRITING
var item = new myRaptorDbEntity();
// Fix #3: Save entire business logic entity
rap.Save(item.ID, item); 

// READING (Every item)
// Fix #4: Type <T> in Query changed from myRaptorDbEntity to mySchemaForRaptorDb 
// Issue #1: Read backs are NOT in business logic objects??
// Issue #2: (s => true) was failing "Stack is empty" exception?!
var q = rap.Query<mySchemaForRaptorDb>(String.Empty); 
var qr = q.Rows;
if (qr != null)
{
    foreach (var item in qr)
    {
        // do work with the schema object i.e.
        // item.SchemaEncShortMsg !!
    }
}
Jan 24, 2014 at 4:42 AM
The following concepts should help you:
  • views are for querying data extracted from document entities in a flat 2 dimensional "table"
  • when querying a view you get back the data in that view not the document
  • You can FullTextSearch() the original document (text only search on the json representation), for times you don't have views defined with the needed data, in which case you will get an array of document id's which you can FetchVersion() from (this is for performance reasons).
  • Save() and Fetch() go together and work with documents.
  • "noschema" is a misnomer since you are always going to need a schema for querying data if you want performance otherwise it would be a limited full text search, what the nosql movement advocates is less schema dependency or in other words schema isolation.
You can get back the entity you saved via Fetch().

Internally all the data is saved in json or binary json based on a configuration setting.

RegisterView() is server/embedded only method and does not exist in the client interface (which is for saving, fetching and querying) so IRaptorDB contains only the methods which exist on the server and client interfaces.

String.Empty is a bug which I will fix since I am not checking for it at the moment.
Jan 29, 2014 at 1:24 AM
Edited Jan 29, 2014 at 2:57 AM
I can't use Fetch() to retrieve all saved entries because that means storing all the GUID keys in a database somewhere for future retrieval (the entire point of RaptorDb).

How can I retrieve all the original objects in their original type? Not their JSON form, not their BSON form but in the Type in which I handed them to over to Raptor?

The Views feature is superfluous if I can't even do such a basic operation. In other words, not having the ability that is an instant-fail during NoSQL technology selection - so having views/mappings etc (other features) don't matter at all.

I hope I'm communicating the problem. For now we have to proceed with an alternative technology...
Jan 29, 2014 at 6:21 AM
All views have a docid "column" which is a reference back to the document which that row was derived from, so you can query views and use Fetch() to get the document you want.

Fetch() returns the original Type of your document (not the json or the binary json).