1

July  19,  2012  

Open  source,  high  performance  database  

What’s New in the C#/.NET Driver

Robert Stam Software Engineer, 10gen

2

•  The  C#/.NET  driver  –  Is  wriFen  in  C#  –  Can  be  called  from  other  .NET  languages  

•  Where  to  find  more  informaIon:  –  hFp://www.mongodb.org/display/DOCS/CSharp+Language+Center  

 

3

•  Recent  versions  –  1.4      Introduced  support  for  LINQ  queries  –  1.4.1    Added  support  for  addiIonal  LINQ  queries  –  1.4.2    A  few  bug  fixes  –  1.5      SerializaIon  changes,  new  Query  builders  

•  Upcoming  version  –  1.6      Support  for  new  server  2.2  features  

4

•  Today’s  topics:  –  SerializaIon  (POCOs)  –  Handling  schema  evoluIon  –  The  new  Query  builders  –  LINQ  queries  –  AuthenIcaIon  

5

•  Documents  are  stored  as  BSON  in  the  database  •  When  you  read  a  document  from  the  database  what  do  you  get  back  in  your  C#  program?  

•  You  can  choose  to  get  back  either:  –  A  BsonDocument  –  A  plain  old  C#  object  (POCO)  that  you  defined  

•  SerializaIon  is  the  process  by  which  a  POCO  instance  is  transformed  to  a  BSON  document  and  back  

6

•  An  in  memory  representaIon  of  a  BSON  document  •  It’s  not  the  original  binary  BSON,  it’s  the  result  of  decoding  the  binary  BSON  

•  Very  similar  to  a  DicIonary<string,  BsonValue>  •  Important  classes:  

–  BsonValue  (abstract  base  class)  –  BsonInt32,  BsonInt64,  BsonString,  BsonDateTime,  …  –  BsonDocument,  BsonArray  

•  More  informaIon:  hFp://www.mongodb.org/display/DOCS/CSharp+Driver+Tutorial#CSharpDriverTutorial-­‐BsonValueandsubclasses  

7

•  Very  likely  you  will  want  to  use  your  own  domain  classes  instead  of  BsonDocument  

•  SerializaIon  makes  that  possible  •  Sample  code:   var collection = database.GetCollection<Employee>("employees"); var employee = collection.FindOne( Query<Employee>.EQ(e => e.EmployeeNumber, 1234)); employee.Salary += 1000;

collection.Save(employee);

8

•  The  driver  users  serializers  to  convert  POCOs  to  and  from  BSON  documents  

•  A  serializer  is  a  class  that  implements  IBsonSerializer  •  A  serializer  is  registered  by  calling  BsonSerializer.RegisterSerializer  

•  The  driver  provides  many  serializers  for  common  .NET  types  

•  You  can  write  your  own  if  you  need  to  •  Class  map  based  serializaIon  works  automaIcally  for  your  POCOs  if  you  follow  a  few  simple  rules  

9

•  If  your  POCOs  follow  these  rules  they  can  be  serialized  automaIcally  by  the  C#/.NET  driver:  –  Has  a  public  no-­‐argument  constructor  –  Has  a  public  get/set  property  for  each  value  that  you  want  to  have  serialized  

•  In  most  cases  automaIc  class  map  based  serializaIon  requires  you  to  do:  NOTHING!  

•  You  only  need  to  do  something  when  you  want  to  override  the  automaIc  serializaIon  

10

•  One  way  to  configure  automaIc  serializaIon  is  by  annotaIng  your  class  with  aFributes  

[BsonIgnoreExtraElements] public class Employee { [BsonId] public ObjectId EmployeeNumber { get; set; } [BsonElement("nm")] public string Name { get; set; } [BsonDateTimeOptions(DateOnly = true)] public DateTime DateOfBirth { get; set; } [BsonRepresentation(BsonType.Int64)] public int Salary { get; set; } }

11

•  If  you  want  your  domain  classes  to  be  independent  of  your  persistence  layer  you  can  configure  serializaIon  in  code  instead  of  using  aFributes  

BsonClassMap.RegisterClassMap<Employee>(cm => { cm.AutoMap(); cm.SetIgnoreExtraElements(true); cm.SetIdMember(cm.GetMemberMap(c => c.EmployeeNumber)); cm.GetMemberMap(c => c.Name).SetElementName("nm");

cm.GetMemberMap(c => c.DateOfBirth) .SetSerializationOptions( DateTimeSerializationOptions.DateOnlyInstance); cm.GetMemberMap(c => c.Salary) .SetRepresentation(BsonType.Int64); });

12

•  Common  changes  to  your  schema  –  You  added  a  new  property  –  You  removed  an  exisIng  property  –  You  renamed  an  exisIng  property  –  You  changed  the  representaIon  of  an  exisIng  property  –  You  changed  the  type  of  an  exisIng  property  

•  MigraIon  strategies  –  All  at  once  using  an  upgrade  script  (much  easier)  –  Incremental  

13

•  ExisIng  documents  don't  have  a  value  for  the  new  property,  new  documents  automaIcally  will  

•  When  you  deserialize  a  document  that  doesn't  have  that  element,  the  property  in  your  class  will  be  null  (or  zero)  

•  You  can  provide  a  default  value  for  missing  elements  if  you  want  

[BsonDefaultValue(Status.Active)] public Status EmployeeStatus { get; set; } // or cm.GetMemberMap(c => c.EmployeeStatus) .SetDefaultValue(Status.Active);

14

•  ExisIng  documents  sIll  have  an  element  for  the  property  that  you  removed  

•  An  excepIon  will  be  thrown  when  a  document  containing  the  removed  property  is  deserialized  because  we  don't  know  what  to  do  with  the  unexpected  element  

•  You  could  handle  this  in  two  ways  –  Ignore  the  extra  element  –  Add  an  ExtraElements  property  to  your  class  

15

•  If  you  just  want  to  rename  the  property  in  your  code  but  keep  the  same  element  name  in  the  database  use  the  [BsonElement]  aFribute  or  the  SetElementName  method  

•  If  you  use  a  migraIon  script  to  update  the  enIre  collecIon  at  once  simply  change  the  name  of  the  element  in  the  database  

•  Otherwise,  consider  wriIng  a  custom  serializer  or  perhaps  just  implemenIng  ISupportIniIalize  

16

public class C : ISupportInitialize { [BsonExtraElements] public BsonDocument ExtraElements; public void BeginInit() { } public void EndInit() { // check ExtraElements for elements with old names // and load them into the appropriate property // also clear them out of ExtraElements so they won't // be saved back to the document again } }

17

•  If  the  C#  type  didn't  change  and  the  two  representaIons  are  compaIble  you  don't  need  to  do  anything.  Old  documents  can  sIll  be  read  and  new  documents  will  be  wriFen  with  the  new  representaIon  

•  Otherwise,  you  will  either  have  to  write  a  custom  serializer  or  a  migraIon  script  

18

•  If  the  new  type  is  compaIble  with  the  old  type  (e.g.  you  changed  from  Int32  to  Int64)  you  don't  need  to  do  anything  

•  Otherwise,  you  will  have  to  write  a  migraIon  script  or  a  custom  serializer  

19

•  What  is  a  query?  –  Anything  that  implements  IMongoQuery  –  IMongoQuery  has  an  implied  contract:  that  when  the  object  is  serialized  to  a  BSON  document  it  will  be  a  valid  MongoDB  query  

•  Ways  to  create  query:  –  new  QueryDocument  {  …  }  –  new  QueryWrapper(object  query)  –  Use  the  untyped  query  builder  –  Use  the  typed  query  builder  

20

•  Version  1.5  introduced  a  new  query  builder  •  What  was  wrong  with  the  old  query  builder?  

–  It  exposed  too  many  of  the  idiosyncrasies  of  the  naIve  MongoDB  query  language  (implied  and,  unusual  restricIons  of  not,  etc…)  

–  It  required  some  helper  classes  (QueryComplete,  QueryNot,  etc…)  that  were  intended  to  be  internal  but  yet  had  to  be  public  or  you  couldn't  write  a  query  

–  The  query  language  rules  that  the  helper  classes  were  aFempIng  to  enforce  were  changing  from  version  to  version  of  the  server  but  the  driver  couldn't  adapt  gracefully  because  the  rules  were  encoded  in  source  code  

21

•  Works  at  a  slightly  higher  level  (e.g.,  requires  Query.And  instead  of  implied  and)  

•  Much  simpler  API  (and  no  helper  classes  that  leak  out  into  the  public  API)  

•  Mostly  compaIble  with  the  old  query  builder  (specially  for  simple  queries)  

•  Easier  to  maintain  from  version  to  version  of  the  server  

22

•  Yes!  The  old  query  builder  will  be  removed  in  a  future  release  

•  For  the  Ime  being,  you  can  conInue  using  the  old  query  builder  without  changing  your  source  code  by  adding  the  following  lines  to  the  top  of  your  source  file:  

#pragma warning disable 618 // DeprecatedQuery is marked Obsolete using Query = MongoDB.Driver.Builder.DeprecatedQuery;

23

•  There  is  a  method  in  the  untyped  query  builder  for  every  query  operator  in  the  MongoDB  query  language  

Query.EQ("_id", 1234) Query.EQ("nm", "John Doe") Query.NE("EmployeeStatus", 1) // assumes Status.Active == 1 Query.GT("Salary", 100000)

// are equivalent to new QueryDocument("_id", 1234) new QueryDocument("nm", "John Doe") new QueryDocument("EmployeeStatus", new BsonDocument("$ne", 1)) new QueryDocument("Salary", new BsonDocument("$gt", 100000))

24

•  You  have  to  know  the  element  name  – What  if  you  use  [BsonElement]  to  change  the  element  name?  

– What  if  you  mistype  the  element  name?  

•  You  have  to  correctly  serialize  any  values  yourself  – What  if  you  serialize  the  value  wrong?  – What  if  you  don't  even  know  how  to  serialize  the  value?  

25

•  The  typed  builder  has  the  same  methods  as  the  untyped  builder  but  is  type  aware  and  type  safe  

Query<Employee>.EQ(d => d.EmployeeNumber, 1234) Query<Employee>.EQ(d => d.Name, "John Doe") Query<Employee>.NE(d => d.EmployeeStatus, Status.Active) Query<Employee>.GT(d => d.Salary, 100000) // also equivalent to new QueryDocument("_id", 1234) new QueryDocument("nm", "John Doe") new QueryDocument("EmployeeStatus", new BsonDocument("$ne", 1)) new QueryDocument("Salary", new BsonDocument("$gt", 100000))

26

•  The  typed  query  builder  also  lets  you  write  the  predicate  in  C#  and  it  will  be  translated  to  an  equivalent  MongoDB  query  

Query<Employee>.Where(d => d.EmployeeNumber == 1234) Query<Employee>.Where(d => d.Name == "John Doe") Query<Employee>.Where(d => d.EmployeeStatus != Status.Active) Query<Employee>.Where(d => d.Salary > 100000)

// still equivalent to new QueryDocument("_id", 1234) new QueryDocument("nm", "John Doe") new QueryDocument("EmployeeStatus", new BsonDocument("$ne", 1)) new QueryDocument("Salary", new BsonDocument("$gt", 100000))

27

var query = Query.GTE("x", 1).LTE(3); // becomes var query = Query.And(Query.GTE("x", 1), Query.LTE("x", 3)); var query = Query.Not("x").GT(1); // becomes var query = Query.Not(Query.GT("x", 1)); var query = Query.Exists("x", true); // becomes var query = Query.Exists("x"); var query = Query.Exists("x", false); // becomes var query = Query.NotExists("x");

28

•  LINQ  is  supported  for  queries  only  (although  Query<T>.Where  supports  wriIng  predicates  in  C#  that  can  be  used  with  Update)  

•  You  opt-­‐in  to  LINQ  using  AsQueryable()   var collection = database.GetCollection<Employee>("employees"); var query = from e in collection.AsQueryable() where e.Name == "John Doe" && e.Salary >= 100000 select e; foreach (var employee in query) { // process employee }

29

•  C#  compiler  creates  an  Expression  tree  •  C#/.NET  driver  translates  the  Expression  tree  to  an  equivalent  MongoDB  query  at  run  Ime  

•  Requirements  –  For  each  C#  property  referenced  in  the  LINQ  query  the  driver  has  to  be  able  to  figure  out  the  matching  element  name  in  the  BSON  document  (using  doFed  names  for  nested  elements)  

–  For  each  test  using  those  C#  properIes  the  driver  has  to  be  be  able  to  translate  the  test  into  an  equivalent  MongoDB  query  operator  

30

•  The  primary  goal  is:  we  will  only  support  LINQ  queries  that  have  a  reasonable  translaIon  to  an  equivalent  MongoDB  query  

•  The  reason:  we  want  to  ensure  predictable  performance  from  LINQ  queries  (no  black  magic,  no  surprises)  

•  So,  you  will  not  find:  –  Any  hidden  map/reduce  or  Javascript  tricks  –  Any  hidden  client  side  processing  

•  Online  LINQ  tutorial  at:  hFp://www.mongodb.org/display/DOCS/CSharp+Driver+LINQ+Tutorial  

31

var query = from e in collection.AsQueryable<Employee>() where e.EmployeeStatus == Status.Active select e; // translates to (assuming enum value for Active is 1): { EmployeeStatus : 1 } var query = from e in collection.AsQueryable<Employee>() where e.EmployeeStatus != Status.Active select e; // translates to: { EmployeeStatus : { $ne : 1 } }

32

var query = from e in collection.AsQueryable<Employee>() where e.EmployeeStatus == Status.Active && e.Salary > 100000 select e; // translates to: { EmployeeStatus : 1, Salary : { $gt : 100000 } }

33

var query = from e in collection.AsQueryable<Employee>() where e.EmployeeStatus == Status.Active || e.Salary > 100000 select e; // translates to: { $or : [ { EmployeeStatus : 1 }, { Salary : { $gt : 100000 } } ]}

34

var query = from e in collection.AsQueryable<Employee>() where e.Name.Contains("oh") select e; // translates to: { nm: /oh/s } var query = from e in collection.AsQueryable<Employee>() where e.Name.StartsWith("John") select e; // translates to: { nm: /^John/s }

35

var query = from e in collection.AsQueryable<Employee>() where e.Name.Length == 4 select e; // translates to: { nm: /^.{4}$/s } var query = from e in collection.AsQueryable<Employee>() where string.IsNullOrEmpty(e.Name) select e; // translates to: { $or : [ { nm: { $type : 10 } }, { nm: "" } ] }

36

var query = from e in collection.AsQueryable<Employee>() where e.Name.ToLower() == "john macadam" select e; // translates to: { nm: /^john macadam$/is }

37

var query = from e in collection.AsQueryable<Employee>() where e.Skills[0] == "Java" select e; // translates to: { "Skills.0" : "Java" } var query = from e in collection.AsQueryable<Employee>() where e.Skills.Length == 3 select e; // translates to: { Skills : { $size : 3 } }

38

var query = from e in collection.AsQueryable<Employee>() where e.Address.City == "Hoboken" select e; // translates to: { "Address.City" : "Hoboken" }

39

var states = new [] { "NJ", "NY", "PA" }; var query = from e in collection.AsQueryable<Employee>() where states.Contains(e.Address.State) select e; // translates to: { "Address.State" : { $in : [ "NJ", "NY", "PA" ] } } // alternative syntax using C#/.NET driver "In" method var query = from e in collection.AsQueryable<Employee>() where e.Address.State.In(states) select e;

40

var desiredSkills = new [] { "Java", "C#" }; var query = from e in collection.AsQueryable<Employee>() where e.Skills.ContainsAny(desiredSkills) select e; // translates to: { "Skills" : { $in : [ "Java", "C#" ] } } var query = from e in collection.AsQueryable<Employee>() where e.Skills.ContainsAll(desiredSkills) select e; // translates to: { "Skills" : { $all : [ "Java", "C#" ] } } // note: ContainsAny and ContainsAll are defined by the C#/.NET

driver and are not part of standard LINQ

41

var query = from e in collection.AsQueryable<Employee>() where e.Addresses.Any(a => a.City == "Hoboken" && a.State == "NJ") select e; // translates to: { "Addresses" : { $elemMatch : { City : "Hoboken", State : "NJ" } } }

42

•  You  can  "Inject"  naIve  MongoDB  queries  into  a  LINQ  query  if  you  need  to  include  a  test  that  LINQ  doesn't  support,  without  giving  up  LINQ  enIrely  

var query = from e in collection.AsQueryable() where e.Salary > 50000 && Query.NotExists("EmployeeStatus").Inject() select e;

// translates to: { Salary : { $gt : 50000 }, EmployeeStatus : { $exists : false } }

43

•  Turn  on  authenIcaIon  using  mongod  -­‐-­‐auth  •  AuthenIcaIon  is  at  the  database  level  •  Add  a  username/password  to  each  database  that  you  need  to  access  

•  AuthenIcaIng  against  a  username/password  in  the  admin  database  is  like  root  access,  it  gives  you  access  to  all  databases  

44

•  If  you  are  using  the  same  credenIals  with  all  databases  you  can  simply  set  the  default  credenIals  

 // on the connection string var connectionString = "mongodb://user:pwd@localhost/?safe=true"; var server = MongoServer.Create(connectionString); // in code var settings = new MongoServerSettings() { DefaultCredentials = new MongoCredentials("user", "pwd"), SafeMode = SafeMode.True }; var server = MongoServer.Create(settings);

45

•  If  you  are  using  different  credenIals  for  each  database  you  can  use  a  credenIals  store  to  hold  all  the  credenIals  (in  code  only,  not  supported  on  the  connecIon  string)  

var credentialsStore = new MongoCredentialsStore(); credentialsStore.AddCredentials( "hr", new MongoCredentials("user1", "pwd1")); credentialsStore.AddCredentials( "inventory", new MongoCredentials("user2", "pwd2")); var settings = new MongoServerSettings { CredentialsStore = credentialsStore, SafeMode = SafeMode.True }; var server = MongoServer.Create(settings);

46

•  You  can  also  postpone  providing  the  credenIals  unIl  you  call  GetDatabase  

var credentials = new MongoCredentials("user", "pwd"); var database = server.GetDatabase("hr", credentials);

47

•  To  authenIcate  using  the  admin  database  you  have  to  flag  the  credenIals  as  being  admin  credenIals  

•  You  can  either  add  "(admin)"  to  the  end  of  the  user  name  or  set  the  Admin  flag  to  true  

var cs = "mongodb://user(admin):pwd@localhost/?safe=true"; var server = MongoServer.Create(cs); var adminCredentials = new MongoCredentials("user", "pwd", true); var settings = new MongoServerSettings { DefaultCredentials = adminCredentials, SafeMode = SafeMode.True }; var server = MongoServer.Create(settings);

48

•  You  have  to  provide  admin  credenIals  to  run  a  command  against  the  admin  database  

var adminCredentials = new MongoCredentials("user", "pwd", true); var adminDatabase = server.GetDatabase("admin", adminCredentials); var adminCommand = new CommandDocument { … }; // some admin command var result = adminDatabase.RunCommand(adminCommand);

49

•  Some  helper  methods  require  admin  credenIals   var adminCredentials = new MongoCredentials("user", "pwd", true); var result = database.RenameCollection( "from", "to", adminCredentials);

50

   

Open  source,  high  performance  database  

Q&A

Robert Stam Software Engineer, 10gen