When connecting to Microsoft Analytics Platform System (APS) from SQL Server Data Tools for Visual Studio 2013

I like to keep it simple so I won’t dedicate much space and time to describing how baffling and annoying this was to solve.

OK, that was a lie; I will have a quick rant.
It seems that the “reward” for keeping it Microsoft (i.e.: building your data warehouse on Microsoft technologies instead of linux/open source alternatives) is hours of installation/configuration time (and this is just to connect from the client, I am not ranting about setting up a server here), and an updated (read: inferior) user interface (there are a lot of annoying differences between keyboard shortcuts for SQL Server Studio and SQL Server Data Tools for Visual Studio – I’m sure this could have been avoided).

Microsoft: you can do better (and you know it).

Anyway, here’s the error message:

The ‘ToolboxPackage’ package did not load correctly.

The problem may have been caused by a configuration change or by the installation of another extension. You can get more information by examining the file ‘C:\Users\$user\AppData\Roaming\Microsoft\VisualStudio\12.0\ActivityLog.xml’.

 

Checking said activity log reveals this:

<entry>
<record>956</record>
<time>2016/01/14 17:17:34.102</time>
<type>Error</type>
<source>VisualStudio</source>
<description>No InprocServer32 registered for package [Microsoft.VisualStudio.Toolbox.ToolboxPackage]</description>
<guid>{6F453E1A-31DF-4CD6-AF8B-72AE242272B9}</guid>
<errorinfo></errorinfo>
<path>%windir%\system32\mscoree.dll</path>
</entry>

Cue lots of googling…

I tried this, this, and this before I found this – which did the job.

The fix was to issue this at the command prompt:

devenv.exe /resetuserdata

 

 

It seems that my user profile was corrupted, resetting my user profile made the error message go away, and left me free to do what I actually wanted to do with APS.

 

 

 

 

Advertisements

When writeBatch to HBaseTable generates OOM (Out Of Memory exception)

Whenever I start to work with a new technology stack I spend the first few years knee-deep in doubt.

Not “why isn’t this thing working?” doubt. That kind of doubt comes after a few years of experience, when I’m sure that what I am trying to do should work.
The first few years are full of “I wonder what this thing is doing?” doubt, or “I wonder if I’m using this thing in the right way?” doubt.

 

Today, a spark job (that batched data into a Hbase table) threw an “Out of memory” exception after inserting about 30% of the desired payload.
The knee-jerk response was “throw some more hardware at it”, but after a bit of tinkering a few discoveries were made: the most important of these was that the HTable class does some interesting things internally with threads – it appears to create a thread for each HTable.put() — and that said threads are not GC’d until you call the HTable.close() method.

 

Today’s solution was to get a BufferedMutator for the Hbase table and to explicitly call .flush() and .close() after every write.

	public void writeBatch(List<Put> puts) throws IOException {
		final BufferedMutator bufferedMutator = tableConnection.getBufferedMutator(tableName);
		bufferedMutator.mutate(puts);
		bufferedMutator.flush();
		bufferedMutator.close();
	}

This probably is not the correct way to do this, but it delivered the expected result (comment if you can see something glaringly wrong).

 

When trying to “Hello World” with mono/.net 4.5, MVC5 in Xamarin studio on Mac OS X 10.10 (Yosemite)

Xamarin provides a “Hello World” of sorts with its vanilla asp.net MVC solution. And, naturally, it works. It also cheats because it doesn’t use a model, it’s just a controller with a view.

So, I decided to try this “Hello World” example by Chad Lung, this one uses a model. And, naturally, this one generates an error message.

System.TypeLoadException
Could not load type ‘System.Web.UnvalidatedRequestValuesBase’ from assembly ‘System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a’.

Description: HTTP 500.Error processing request.
Details: Non-web exception. Exception origin (name of application or object): System.Web.Mvc.
Exception stack trace:
at System.Web.Mvc.FormValueProviderFactory.GetValueProvider (System.Web.Mvc.ControllerContext controllerContext) [0x00000] in :0
at System.Web.Mvc.ValueProviderFactoryCollection.GetValueProvider (System.Web.Mvc.ControllerContext controllerContext) [0x00000] in :0
at System.Web.Mvc.ControllerBase.get_ValueProvider () [0x00000] in :0
at System.Web.Mvc.ControllerActionInvoker.GetParameterValue (System.Web.Mvc.ControllerContext controllerContext, System.Web.Mvc.ParameterDescriptor parameterDescriptor) [0x00000] in :0
at System.Web.Mvc.ControllerActionInvoker.GetParameterValues (System.Web.Mvc.ControllerContext controllerContext, System.Web.Mvc.ActionDescriptor actionDescriptor) [0x00000] in :0
at System.Web.Mvc.Async.AsyncControllerActionInvoker+c__DisplayClass21.b__19 (System.AsyncCallback asyncCallback, System.Object asyncState) [0x00000] in :0

Thoughts.
I’m sure that asp.net error messages could be less terrifying.
I wonder how many other people are using Mono/Xamarin/asp.net MVC on Mac OS X 10? Probably about 3.

I’ve googled the error. Found this: http://curtis.schlak.com/2014/02/04/setup-asp-net-mvc-4-on-monodevelop-4.2.html
Written over 12 months ago and it appears that things have moved on since then (assumption I came too after noticing that
my menu options are slightly different to the ones described in this post).
I have also seen that some people suggest copying a different version of a .dll into my Xamarin solution. Pretty terrifying, considering that my expected result here is a web page with “Hello world” written on it.

Others suggest downgrading to version 4 of asp.net MVC. This sounds like the most “solution-that-I’m-willing-to-try-able” solution.

It’ll have to be another night, though, as I’m off to bed now (completely wasted couple of hours… bloody asp.net).

Togetherness AKA: simple testing can prevent most critical failures

One of the things I love about my profession is that it inspires empathy for people that I have never met, never spoken to or even exchanged emails with.
How? Through code comments (that’s how).

Code comments are generally short and acerbic.
On occasion they are philosophical.
And in the best cases they reveal a profound sense of confusion and utter desperation.

Here are some gems from Ding Yuan’s talk on testing in large scale distributed systems.

catch (NoTransitionException e){
// why can this happen? Ask God not me.
}

This one’s from an optimist.

/* if this happens, hell will unleash on Earth. */

It almost certainly isn’t.

/* TODO: are we sure this is OK? */

This one’s from a realist.

/* TODO: this whole thing is extremely brittle */

There are a few more of these, and a great deal of fascinating insight into the benefits of static review in Ding Yuan’s complete talk.

Prepare for glory, 2015.

Close your eyes.
Make a wish.

Here are some (perverse, and in some cases: sadistic) things that I hope that no one will say this year. Or any other year. Ever again.

“We just want a few simple pages with a few links, there’s no need for things like testing.”

“We don’t want to get into a situation where we have to spend lots of time writing things down, because we haven’t got time.”

“This is going to be a quick and straightforward thing, there’s no need for documentation.”

“Can we just get something up and running quickly and then do all of this other stuff (read: testing) a bit later on?”

“Can we not just keep track of things via email?”

“Sorry but we can’t say ‘no’ to him/her.”

Posting now, but I’m sure I’ll think of more later on today …

Doncs, que ha passat?

https://adventuresintesting.wordpress.com/about/

“If fate permits, this blog will also allow me to practice writing in Spanish and/or Brasilian Portuguese.

If fate is in a really good mood, I may even be able to practice writing in Catalan.
A veure…”

Aleshores, que ha passat?? Com pot ser que porto no se quants mesos escrint exclusivament en angles??  Estic fatal, FATAL!

Had to get that out of my system.
And I have to make more of an effort to practice these languages!