Deleting and storing files/binary data

Nov 14, 2012 at 6:51 AM

Hi,

I have been talking to you on codproject about the query syntax but i am finding codeproject a bit of a pain for discussions so i thought i would try here.

 

One other thing i am considering using this database for is cache. A file cache for one of my http libs in my app to be exact. That then raises 2 questions.

1. Can you explain deleting and why it is the way it is? The codeproject article makes it sound like deleting and entity does not delete it which is worrying for a cache as it would forever grow and i could not get rid of data i don't want any more.

2. What would be the recommend way to store binary data plus a little meta data such as name etc that would be used to extract the binary data.  My files should not be bigger than 5MB but i want to understand how this part works and best practice so i don't start with a  bad design. Most of my files will be under 100KB but i will have many of them and don't really want a file on disk for each of them if possible. 

 

There are other solutions i can consider it is just that since i am already using your database it seemed a good idea to use it for the cache as well if it will support it.

Chris 

Coordinator
Nov 14, 2012 at 2:14 PM

Hi,

1) Delete operations are costly and require a lot of code to implement (track, defrag free spaces etc.) and the storage files were implemented as append only, and in my experience delete operations are not really suited for business apps were you can defend your position by showing the chronological order of events. The only way is to copy your data to a new database file if you want to compact things (it is trivial to do so and maybe I will add a "compact" method in the future).

2) Binary data can be stored by using the SaveBytes() and FetchBytes() [these are stored in a separate storage file to "data" objects].

Nov 14, 2012 at 7:23 PM

Fair enough.  For now i think i will use http://filedb.codeplex.com/ for storing my files since i need cache system for them and they will change a lot or be different files.  I will still use your db for my data just not the files.

 

Adding a compact method to your database could be useful for people that don't want to keep deleted items and anything else that uses space but is no longer the latest version. If you were to add that method i would suggest adding it so that is can be run at the same time the database is in use so that it could be run as a background task on another thread with the understanding it will take a while and may slow down that database a bit while it is running but the database will still function.