May 2008 Entries
I have been waiting for this, for oh... 25 years... While it may not seem remarkable in the demo, having it built-in to the OS is the key to its success. We can do most of this today with add-ons, but they don't interact well with older Apps, just try changing the Font DPI in Vista and you will see what I mean. I hope this is taken into account in the next generation of our UI. The Mouse is a stupid device, but nothing has been able to provide its functionality yet, multitouch can, if done right.
Can I have my 19" Desktop replacement with Multitouch NOW please?
I can't wait to start working with this...
Ballmer And Gates Demo Windows 7: Multitouch, Pie Menus And a Piano | Gadget Lab from Wired.com
Now for the 2 year wait until its release :-( working with these silly restrictions of UI we have today.
I am about ready to STOP doing WebApp development and move everything to WPF and Silverlight, but not quite yet.
I really like what the good Doctor is talking about here.
Social networking should lead us to better in depth personal communications, not expose us to a ton of noise and frustration through outages. It MUST work with my phone and be incredibly simple to communicate, today, its not.
The problem is, this is just a "teaser" of what it really could be, and it's totally disconnected from all the other "social" apps. It doesn't connect you to opening an IM chat (which is realtime, and often a lot more useful), there's no easy way to make it really integrate into a forum / messageboard (which is more permanent, and searchable), or del.ico.us (which helps with organizing and tagging resources that are shared with others), or a hybrid short url / spidering / social tagging site like my ittyurl.net, or a blog post with comments, or email notification of something I am following and on which I need up - to - date information when new "stuff" appears on the subject.
Peter Bromberg's UnBlog: The Social API we really need.
I also think that Phone software needs a major overhaul to support this type of communications. I am fairly happy with my BlackBerry 8830, I dislike the KB... I am pretty sure I will replace it in the fall, but not until I am resolved hat the next Phone I get will do everything I need it to. The frontrunner is the iPhone for me, but I would prefer a tactile keyboard.
My phone is my main communication device, it should do what I need it to do without a lot of hassle, I sure wish MS could get their act together on Mobile Services with Visual Studio. I also wish there was a common platform that ALL phones worked with, this is currently 10x worse than the Win/Linux/Apple platform problem. No resolution in sight, Silverlight could be the magic bullet if MS can convince Apple and RIM to support it.
I do think that Twitter and the other "social services" should become an RSS redirector that consumes my "social feed" and redirects it to my "followers" in semi-realtime. A few seconds of lag is fine if his means a decentralized posting center. If I could post to a self-controlled (read hosted under my control) feed that all the social networks could consume and broadcast to my "friends" then we would have a more sensible service, providing better scalability and less loss of MY posts.
I have had too many posts lost to Twitter that could be there on my own RSS "social feed" that could be picked up my pownce, jaiku, friendfeed, facebook, twitter and all the others instead of making me cross-post.
We could designate a Category in the RSS for Pulling feeds, or we could use the lesser-known Cloud tag to link in to our chosen social platforms. Comments and solutions welcome.
Yesterday Scott at OdeToCode talked about some good insights about the visualizers in VS08.
Visual Designers Don’t Scale
K. Scott Allen - 5/19/2008 9:32 PM PST
Visual tools that fall into the second category have to cover a wide range of scenarios, and they need to scale. I stumbled across an 8-year-old technical report today entitled "Visual Scalability". The report defines visual scalability as the "capability of visualization tools to display large data sets". Although this report has demographics data in mind, you can also think of large data sets as databases with a large number of tables, or libraries with a large number of classes - these are the datasets that Visual Studio works with, and as the datasets grow, the tools fall down.
Maybe they can add some of DeepZoom's technology here but I am not sure how that could be accomplished easily, Breaking up the Image is the most likely way to accomplish it, create related sections of your Schema in different diagrams, at least that is how I have been working around it in even my smaller applications. But this of course doesn't really help when you are looking at the BIG picture :-)
Reading in the Industry blogs, official samples and documentation provided with MS SQL Server 2005+, Visual Studio 2008 and Linq to Sql and the Entity Framework, you will get the impression that we have a tremendous ability to efficiently decipher how to interact with a Database. Additionally, Industry leaders have led us down a path of Code Generators, Designers and the separation of UI Design from functional Object code; read "Astoria" and/or "MVC" here...
Didn't CASE do this sort of thing for us years ago? Didn't it also fail? There is huge amount of traffic about the topic of "Falling down when you hit a threshold" and insistence that all things should be built ultra-scalable. OK, that's great if you love to write plumbing. I for one, don't. What if that could be done for you, with the most efficient options always at your disposal; of course with the opportunity to tweak and extend to meet a specific need.
While there are many great arguments to be had over what is exactly ultra-efficient and those who think they have a better method must be able to override and extend the selected generation of code that is offered by not only being able to write that code, but also to inject it into the Generated Code (or at the least allowed to edit the templates) produced by Visual Studio.
If you don't already know why this is so important, we need to first look at the massive shift in Business Software Development and where we are going to be running most business applications. We used to run all our apps on our own hardware, inside our own network and everything was under our tight control. While this model still applies to much corporate development, I believe it will become less and less used as a cost effective alternative to a HOSTED solution. Pricing of hosted solutions have plummeted in recent years, in fact I am almost willing to bet that the cost of electricity alone using hardware just a few years old would cover the cost of a hosted solution. And what happens when I DO need to scale, How do I add 20 servers to the Web Farm overnight? Staffing requirements and maintenance of hardware start to become totally offset when we look at this model. I CANNOT BUILD A SERVER CHEAPER THAN I CAN HOST ONE! That is the real bottom line.
I would wager that MOST applications would run just fine on a hosted solution with one machine doing all the work, at least in the beginning, or at least with a hoster that gives you Web on one machine and your DB on another. Are we talking about every solution here? No, but I think we are talking about a gigantic sector of the market that is currently all but ignored by the provided tools we have at our disposal.
Don't get me wrong here, we have GREAT tools available and more coming, do they target what is really needed?
First lets look at the three new primary tools given to us by Microsoft. WCF, Linq to Sql and the Entity Framework. WCF provides a great pipeline to pass data between our UI, Business Objects and Database, but it just plumbing. Linq to Sql and EF provide a great way to interact with a Database IF and ONLY If you have a proper database in place first.
What does this mean to most data-driven development projects? It means they either need a DBA, or they need to hire one for the portion of the application that defines the database. This is one reason Migrations for Ruby on Rails has gained so much popularity, it hides this chore. Most of us don't care about the DB, it's just a place that holds data. Some of you can go on and on all day about the subtleties of databases but that is completely outside the scope of what I am talking about here. Not only will Migrations create a DB and Schema around what your App needs, it will also keep it up to date with changes made by any developer that can be easily stored in Version Control software like SVN or TFS.
This is what the Designers for Linq to Sql and/or Entity Framework need to do for us. I really don't mind changing my Linq to SQL Apps over to Entity Framework if it will allow us to focus on one great technology. It's great that they can operate against a currently created database, but they should also create one (IN AN OPTIMIZED WAY) when one does not exist. It should create Stored Procedures if I tell it to... Even if we are migrating a legacy application, I would still like to see what a tool spits out for me. I just might use the new techniques. There are tons of ways to move data from one schema to another, again, outside the scope of this article. So back to the scope... EF Does a great job of creating an interaction between Database and Business Objects, but it doesn't help us rapidly create a NEW model without an existing database and I think this is important in two ways, first and foremost, I don't want to write any plumbing code to do this. The tools are smart enough already to do almost all the work for us, it can go just a little bit further and create the expected interaction that solves most of the interaction of the database work for us. I can create a Model with the EF Designer by adding Properties and Associations, but it won't go and create a database around that model.
I should not have to know about how the interaction happens, just that it does and that it does so in a fairly efficient manner. If I lose a certain percentage of performance due to layers, then so be it if I can create my application 10 times faster. If the tool developers choose to implement change tracking in a DataContext or Object Context that is great, do so in a way that makes sense for the CODE developers to use, not require me to go hire Database professionals to argue about the proper plumbing or storage techniques. Linq ALREADY knows how to create efficient SQL, if it did so in code generation vs. dynamically at runtime who cares?
We can host a solution out in the cloud so should we need to be worried about the optimization of how our application persists its data? Certainly there is is need to cover legacy applications and all the DBAs running around out there, that is well covered right now. What we seem to be missing is the ability for competent application developers to create Rich Internet Applications (read Silverlight and Ajax) and have the data persisted without a bunch of plumbing and tweaking.
What is NOT covered is the ability to rapidly envision, design, develop and deploy solutions destined to live on a Hosted Solution -- even if it is "hosted" internally. This solution will most likely have a distinct separation between UI, Business Objects and Database, but do I really need to see those distinctions in my code? Not really. Do I really need to spend the time to wire up all the Databinding when the MetaData already knows what I need to do? No Way, a tool can create that for us. It can already look into our metadata and see what we need to do to hook up a DataGridView to a table.
I would venture to say that the tools are already here, they just don't work the way we need them to so we can build these applications quickly. I spend MOST of my time Creating a Database, then working on plumbing and interactions between the Database and the UI, I should not have to do this any more. Microsoft did a fabulous job of giving us a database that can scale with us as our application grows. We have SQL CE, SQL Server Express and SQL Server Enterprise to grow with us, we can use SQL CE for Mobile Apps and offline stuff, wouldn't it be wonderful if I did not have to architect that portion of the solution any more? I am talking about the people who do not have the luxury of a full time DBA.
Of course, the solution is there, staring us right in the face, the answer is the Designer for Entity Framework. If this Designer could be used to build out the Database, and track the Schema changes, then it would take down part of the battle, just a little more work and it could completely remove the NEED (we should always have the option to extend and modify though) for us to write anything between the Business Objects and the Database.
We should not need to write Serializers for Silverlight, WCF or anything else, the tools already KNOW how to do that, they just don't. The MetaData is there, let's use it. Rob Conery's Subsonic and ActiveRecord shows there is interest in such a tool, but I say this tool needs to come from Microsoft and it needs to be built into the current generation of tools we are using or are being pushed to use (aka Linq). With Rob and Phil working AT Microsoft why can't they contribute some of that knowledge over to the ADO team and get this into the EF Tools we need. Pablo, Mike, Dinesh and the rest of the ADO Team are doing a great job so far, but they are only touching on what we really need. The newest version of EF finally gets a way to do disconnected graphs which is a step towards the goal thanks to Danny Simmons.
These tools need to work seamlessly with Linq and allow me as a developer to write applications with little or no concern for how my data is persisting. I want to create Controls that I can offer to my UI which work seamlessly with Ajax or Silverlight and not require me to write a ton of plumbing just to move things back and forth from one layer to another.
This is the challenge for the new world of development. We have already moved to a hosted world and our tools need to move there with us. But what do I know, I am a VB developer first and foremost... This is what I know, my customers are asking me to create these applications and to create them really fast. I can choose to use the tools I know best or I can branch out and find something new. The problem here is that I have already invested years of time to learn the tools I am using and they are simply not doing what I need them to do to make my job truly easier.
My Invoice Object doesn't really need to know what DataContext my Customer Object lives in and I as a developer should not have to be concerned about that in the slightest. I should be able to freely take a Customer I loaded from Invoice "A" and assign it to be used for Invoice "B" without worrying what context it came from, Invoice "B" just needs to know its a valid Customer or that it is a new one I just created without me forcing something into the plumbing to test it. It should know how to roll itself back when I cancel changes, etc.
With Generics and XML Literals, VB is a Top Notch Code Generator. Kathleen has proven that here and Karl talks about Metadata and what we can do with it over here... We don't need Yet Another Code Generator. We need something that works with the existing modelers and designers and just makes our life as developers easier. It needs to be INSIDE Visual Studio and we need to be able to work with the templates. I should not need LLBLGen or MyGeneration or CodeSmith or SubSonic to do something Visual Studio pretty much already knows HOW to do, it just doesn't. I don't need another designer that works with an existing database, I need the CURRENT Designers to Make the Database AND the plumbing.
I personally think this should be the direction for the next revision of Visual Basic. VB IS the glue that puts the framework together, sure you could also do it in C# (if they added XML Literals...) but VB is already geared for it so why not enhance it's productivity even further by giving us a way to be far more productive than we are today. This does not require any great shift in the tools we already have, it just requires Microsoft to use them smarter and provide us the keys we need to open the door.
I am working extensively with Linq as well as Silverlight. Unfortunately they are not in the same release cycle :(
So for now, I am back to running my development machine with 2 Hard Drives. (because VMs suck on my laptop...)
Further info here: Error installing Visual Studio 2008 SP1 Beta and Silverlight Tools Beta 1
[total tangent department]
Maybe I can get a new Laptop soon with a 17" screen, Hyper-V and 64Bit OS... and Tablet functionality... not holding my breath... they don't exist. I just might have to build one, but I am not feeling that adventurous this month. Maybe I'll just dedicate a small notebook to notes, but that defeats the purpose for me in most cases.
[/total tangent department]
Roy gives us 2 great things here: An easier to understand Regex for people who don't care to learn the ins and outs of regex AND more importantly an example of how to build a Linq Provider. I may tap into this idea for producing a couple Linq Providers I am thinking about, such as Linq to RSS and Linq to NNTP, alas, I really don't want to be a plumber this month so I am hoping someone beats me to the idea for Linq to NNTP which I could really use :-)
Introducing LINQ To Regex
When I first saw Josh Flanagan's Readable Fluent Regex Api, I loved it at first sight. Now I'm working on a demo for the upcoming DevTeach in Toronto, where I'll present how to build custom LINQ providers to various things.
I thought it would be interesting to see if using the fluent Regex API Josh created along with a LINQ query syntax would "work" for me, so I set out to create LINQ to Regex.
Introducing LINQ To Regex on Roy Osherove's Blog
Visual LINQ Query Builder is an add-in to Visual Studio 2008 Designer that helps you visually build LINQ to SQL queries. Functionally it provides the same experience as, for instance the Microsoft Access Query Builder, but in the LINQ domain. The entire UI of this add-in uses Windows Presentation Foundation. The goal of this tool is to help users become more familiar with the LINQ syntax. The tool may also demonstrate to users how to create their own Visual Studio 2008 add-in using Windows Presentation Foundation.
This is very helpful in letting you see the big picture when you have particularly complex queries. I'd really like to see this integrated with something like LinqPad so we could get a better look at how we can structure Linq Queries. I'd also like to see this not need to add something to your Project just to use it, that is a real turn off for me, but you can always delete it when you are done. This is a good effort though on helping to simplify understanding new technologies. I'd also like to see it brach out to support more than Linq to Sql.
On the positive side, this does have a pretty good visual appearance in VS08 and I like how it is put together, building queries is pretty easy to do with the leveraging of intellisense. Some big missing items re Grouping and Aggregates, those tend to be the more difficult queries to visualize and where this tool would be most useful.
I would place this in the beginner category though because it will tend to get in the way of a seasoned Linq developer.
While I am doing this in a VERY Different way... it is worth mentioning the Unplugged LINQ to SQL Generator which is available on CodePlex:
http://pampanotes.tercerplaneta.com/2008/03/implementing-n-tier-change-tracking.html
When designing our application data model, we think some of our entities as an in-memory cache of small pieces of data living in a database (MSSQL, Oracle, Xml files, etc.). This data will be jumping between both worlds.
To keep this "cache" synchronized, we need mechanisms to read data from storage, and update changes back, to perform that we create CRUD operations.
As Chris Rock pointed out, the first release of our Code Project custom tool for LINQ to SQL code generation lacks any usage documentation.
I am not sure this is the approach I want to take, but it does have some interesting possibilities.
I am in the process of writing up the way I am approaching this EXACT SAME DILEMNA in a different way; I am using inheritance of Generic Base Class to accomplish pretty much the same thing. The difference in my approach is that there is no need to do any CodeGen outside the regular way that Linq To SQL already does it, it's just there. This also has pluses and minuses so maybe a combination of the two is in order.
Of course, Mine is also in VB not just C#...
mattwar tell us the proper way to0 "extend" DataContext in Linq to SQL.
"I often get asked how LINQ to SQL is supposed to be used with Test Driven Design (TDD).
LINQ to SQL was actually designed to be host to more types of back-ends than just SQL server. It had a provider model targeted for RTM, but was disabled before the release."
I am still reading this, some of it looks good, some of it looks not so good...
DISCLAIMER: Overriding internal implementation details is not a practice recommend or supported by Microsoft. Implementation details are subject to change without warning.
I am not sure that extending the DataContext is really what I need to do for my "Entities with State" model, but I am willing to look at it in that respect.
I think I still prefer what I am doing, ignoring most of the stuff the DataContext tries to help with (in a totally connected type of way...) and move those things to the Entity so I can be completely disconnected much easier.
I needed to find out exactly what was being submitted by DataContext.SubmitChanges and didn't want to use SQL Profiler.
I ran across this post by Kris Vandermotten and have converted the code to VB for anyone else who may need it.
This is now always in my Utility.vb file that gets attached to most of my projects.
Imports System.Diagnostics
Imports System.Globalization
Imports System.IO
Imports System.Text
''' <summary>
''' Implements a <see cref="TextWriter"/> for writing information to the debugger log.
''' </summary>
''' <seealso cref="Debugger.Log"/>
Public Class DebuggerWriter
Inherits TextWriter
Private _isOpen As Boolean
Private Shared _encoding As UnicodeEncoding
Private ReadOnly _level As Integer
Private ReadOnly _category As String
''' <summary>
''' Initializes a new instance of the <see cref="DebuggerWriter"/> class.
''' </summary>
Public Sub New()
Me.New(0, Debugger.DefaultCategory)
End Sub
''' <summary>
''' Initializes a new instance of the <see cref="DebuggerWriter"/> class with the specified level and category.
''' </summary>
''' <param name="level">A description of the importance of the messages.</param>
''' <param name="category">The category of the messages.</param>
Public Sub New(ByVal level As Integer, ByVal category As String)
Me.New(level, category, CultureInfo.CurrentCulture)
End Sub
''' <summary>
''' Initializes a new instance of the <see cref="DebuggerWriter"/> class with the specified level, category and format provider.
''' </summary>
''' <param name="level">A description of the importance of the messages.</param>
''' <param name="category">The category of the messages.</param>
''' <param name="formatProvider">An <see cref="IFormatProvider"/> object that controls formatting.</param>
Public Sub New(ByVal level As Integer, ByVal category As String, ByVal formatProvider As IFormatProvider)
MyBase.New(formatProvider)
Me._level = level
Me._category = category
Me._isOpen = True
End Sub
Protected Overloads Overrides Sub Dispose(ByVal disposing As Boolean)
_isOpen = False
MyBase.Dispose(disposing)
End Sub
Public Overloads Overrides Sub Write(ByVal value As Char)
If Not _isOpen Then
Throw New ObjectDisposedException(Nothing)
End If
Debugger.Log(level, category, value.ToString())
End Sub
Public Overloads Overrides Sub Write(ByVal value As String)
If Not _isOpen Then
Throw New ObjectDisposedException(Nothing)
End If
If value <> Nothing Then
Debugger.Log(level, category, value)
End If
End Sub
Public Overloads Overrides Sub Write(ByVal buffer As Char(), ByVal index As Integer, ByVal count As Integer)
If Not _isOpen Then
Throw New ObjectDisposedException(Nothing)
End If
If buffer = Nothing OrElse index < 0 OrElse count < 0 OrElse buffer.Length - index < count Then
' delegate throw exception to base class
MyBase.Write(buffer, index, count)
End If
Debugger.Log(level, category, New String(buffer, index, count))
End Sub
Public Overloads Overrides ReadOnly Property Encoding() As Encoding
Get
If _encoding Is Nothing Then
_encoding = New UnicodeEncoding(False, False)
End If
Return _encoding
End Get
End Property
Public ReadOnly Property Level() As Integer
Get
Return Level
End Get
End Property
Public ReadOnly Property Category() As String
Get
Return _category
End Get
End Property
End Class
To use this for troubleshooting DataContext issues simply set the DataContext.Log = New DebuggerWriter such as this:
Dim tran As New Transactions.TransactionScope
Using tran
db.Log = New DebuggerWriter
db.SubmitChanges(ConflictMode.ContinueOnConflict)
'rollback by disposing without complete
tran.Dispose()
End Using
This will submit the changes, spit out all the SQL code to the Debugger Window and roll everything back.
I have been working with linq for a decent amount of time now and there are still some things about it that really bug me...
1. When you have a model that has both ID Fields and an Association (which just seems logical because we don't want to force a load of an object just to look at its key) if you are creating new records and use InsertOnSubmit, if you have an attached Association - Say an Invoice with an attached Customer, even though the Customer is already in the database, Linq to Sql will helpfully just add a new one and create a new ID for you. Holy Duplicate Data Batman! I have no idea why this would seem like a good idea to anyone. If I have a record that is associated and loaded and it has a PrimaryKey defined, why in the world would it even attempt to insert it? This happens even when I loaded the Items with LoadOptions. but I do admit that I am dropping the DataContext in between retrieve and submit, nevertheless, this should not happen.
2. If you need to delete records and submit new ones, there is no guarantee that the deletes will happen first so you are stuck with having to do the deletes first, then add the new ones in a separate command or you will most likely get key violations unless you happen to be extremely lucky on occasion. Thankfully we have TransactionScope to deal with the whole submit as unit of work.
3. Loading an Association does not automatically set the ID if that field is also in the map. This is another weakness that should happen automatically. I should not need to write that myself in OnPropertyChanged.
There are some others, but it is these 3 that continue to annoy me time after time when using Linq to Sql. I keep trying to blame myself about not "getting it right or something". However, I just can't. This seems like a reasonable expectation of a well thought out framework for working with objects, unless of course you like re-routing plumbing to make it go your way.
Blogroll Me!
Copyright © 2003-2009 H. Steele Price, IV -
All opinions are my own, not necessarily those of my employer, your mother, or any government agency.