Learning is Painful

This is a post about my damaged spinal cord, but I’ve been working at Automattic for seven years now, so I really can’t help but think about the first line of the Creed:

I will never stop learning.

I strongly identify with this line, but damn it is hard sometimes. The past two to three months I have been in physically terrible shape. Every 3-4 weeks I’ve been learning something new about how my body now works, and this learning is both metaphorically and physically painful.

Aristotle – “Learning is not child’s play; we cannot learn without pain.”

Damn Greek philosophers!

Really I think I’ve been learning something new about my body every few weeks for the past 10 months. Currently feeling hopeful that I can put some of the physical discomfort behind me, but a part of me knows that is probably overly optimistic.

Abdominal Muscles

In early June I went to a work meetup in Denver which contributed to me pausing my PT exercises for about two weeks. Then I tried to pick up where I had left off. It turns out though that muscles atrophy faster when you have a damaged spinal cord, so two weeks off is more like 4-6 weeks off for me. On top of that, it turns out I don’t really have much sensation in my abs. So when trying to do my PT, I didn’t realize that I had badly pulled my abs, so I kept going and pushing through as my exercises got harder. And then for the next week I really couldn’t stand up. About a month later, as I was finally getting better I pulled them again, probably due to dehydration.

My lack of sensation in my abs is so bad that my PT can push on them to get the muscles to release and I am perfectly comfortable where he says anyone else would be in agony. Turns out pain is a really useful feedback mechanism.

I now finally understand why every few years since about 2006 I have badly screwed up my abs to the point of having trouble standing. I just didn’t have enough sensation to tell myself when I needed to back off from whatever I was doing. This problem has been there for a long time. At least I understand it now.

Sleep Apnea

I started on an APAP machine in mid-June. It wasn’t until about two weeks ago that I found a mask configuration where I am pretty comfortable and feel like I can sleep well. There was a lot of bad sleep along the way to figuring that out. But I think I can now see some improvements to two of the fatigue metrics that I have been tracking: grip strength and heart rate variability. This is pretty exciting. My grip strength – yes grip strength! – has increased something like 20% on average. Similarly my heart rate variability has increased by 10-20% which is an indication of less fatigue.

Pretty exciting that I am seeing an improvement in the metrics that I had identified as mattering. For some reason my watch has not been showing an increase in deep sleep, but overall I have also been feeling somewhat less fatigued so I’ll take it.

Unfortunately I have actually been feeling more tired which is probably related to how much sleep debt I have. Supposedly that can take months to catch up on. Just in time for kiddo number two to show up! 😉


The other recommendation I got to deal with fatigue was to start taking Cymbalta which is both an anti-depressant and can reduce nerve pain (maybe allowing me to reduce another medication which can also add to fatigue). I started in early July, and the side effects have been pretty terrible: insomnia, nausea, dizziness (especially looking at a screen), and light headedness due to my blood pressure dropping.

The first week was so bad that after days of nausea and insomnia I repulled my abs because I got very dehydrated. Then ended up calling an ambulance at 3AM when I couldn’t stand up without getting dizzy. Every time I stood up my blood pressure would drop about 40 pts. After they put two liters of fluid in me I was a lot better, but ya it was a bit exciting.

Sidebar: when you are laying on the floor unable to stand, alone in the house at 3AM (Mekayla and Morgan had gone camping), but otherwise ok, it is really hard to know when to call an ambulance. Also, it is really nice when you have a watch on your wrist from which you can make a phone call rather than having to go find your phone.

After about five weeks on Cymbalta, I think I am mostly adjusted to the medication. I am not sure it has helped with fatigue, but I am in an amazing mood. Given how rough the past month has been, I have actually been feeling pretty good about everything.

So… um… yay anti-depressants. I don’t actually think my depression was all that bad. It happened occassionally. What I thought of as maybe a few times a year. The reality is that it was always probably there more than I even now want to admit. Wil Wheaton probably explained this better than I can. But I would say that even though I am unsure just how bad it was, that I wish I had tried this years ago. If you are reading this and thinking about it, you should know that there are other meds besides Cymbalta where the side effects are typically not as bad.

Increased Sensation

This one is really just funny.

So my spine has become a bit crooked (not yet the funny part). It turns out that this was because my shoulder and back muscles had become chronically tight to compensate for my bad balance. My PT has been working on this for many months, and it is improving, but I decided to also add in getting a massage from someone who has experience with spinal cord injury clients. In lots of places my body has reduced sensation, but the sides of my torso are extra sensitive. A little bit of pain is just unbearable even when I know the massage is not that strong.

So, the massage reminded me that my sides are extra sensitive, and then solved a long running bout of marital discord. For years, Mekayla would lay on my shoulder, and then I would yelp and ask why she was digging her fist into my side and she we would plead complete innocence. Ignorance even. She would complain that I was being overly sensitive and I would say something about why she has to twist her knuckles into my ribs. This has been going on for years.

Turns out I was being overly sensitive.


Gnōthi Seauton


Relearning to Walk at Forty: Fatigue, Physical Therapy, and Data

I spent most of 2017 trying to diagnose the slow degeneration of my ability to run and then walk. Then the past nine months have been slowly trying to discover how much I can recover from my spinal cord injury. My spinal cord damage is non-traumatic which means that it is a bit different than the typical spinal cord injury – though I’ve rapidly learned that all spinal cord injuries are really pretty unique experiences. In general, for someone who is missing 30% of his spinal cord for a couple of vertebras I am doing really well, but that really doesn’t make it feel any easier most of the time.

My rehab is still a work in progress, but I’ve been wanting to write about the data I’m collecting for a number of reasons:

  1. I learn a lot by writing, and it helps get the details clearer in my head when I write them down.
  2. Hopefully sharing this can give ideas to other people struggling with similar problems. I’ve taken a lot of ideas from folks with multiple sclerosis, chronic fatigue syndrome, and of course spinal cord injury.
  3. Comments on blog posts can often provide a lot of great feedback and other ideas of things to pursue, so hopefully I’ll learn something from you.

So I’m going to cover a good chunk of the data that I am tracking about my day to day symptoms. If you don’t want to know a lot of details about what is going on, you should stop reading now. Also, I should probably warn you that I like data a lot, track a lot of data, and this is going to be a 3,000 word post. 😀

Background on my Symptoms

Probably helps to outline my symptoms around walking/moving:

  • Injury diagnosis: T1 ASIA D – Sensation changes start at Thoracic 1 though most of the visible damage on the MRI is from T5-T7. “D” because I can walk, and don’t need a cane or anything. For comparison “B” would be in a wheelchair but with some sensation/control, and “E” is normal.
  • Walking ability: I was close to falling down a lot (and did once), but the Tizanidine I’m taking has gotten the spasticity under control so that I almost always feel pretty stable on even surfaces. Changing speed is challenging though as is looking around while walking, but both are improving. Otherwise my walking is mostly limited by endurance and how much pain I want to be in. I’m averaging about 4.5k steps a day this month. Typically, less than 5k steps I’ll be fine, and more than that I’ll be in pain later in the day or tomorrow.
  • Muscle control and gait: I have poor control over lots of muscles from my chest down, and it is not symmetric. Certain sides and certain muscles are very hard to activate. For example, my left hip is much harder to control than my right. This resulted in some odd walking problems where I’ve learned (over more than a decade) to balance using my left shoulder which is always really tight, and my right hip which is always sore. Was resulting in a slight limp. Also my ankles have very poor control. Manual manipulation by my PT and lots of focus on being able to activate particular muscles is slowly helping. I have a lot of habits to retrain.
  • Pain: My feet are always numb to some extent. As I get more tired it becomes a burning pain and eventually my legs start twitching which also hurts. Overall pain usually varies from 2 to 6. I haven’t not been in pain for over a year. I’m taking Gabapentin to reduce the nerve pain. Helps a lot, especially with falling asleep. Only take it at night because it was increasing my fatigue during the day though so often in more pain as the day goes on and the drugs wear off. Most of my pain doesn’t happen immediately, it kicks in a few hours after the activity, so it is very hard to have good feedback.
  • Fatigue: My fatigue is delayed by many hours and often builds up over the course of multiple days. This is not muscle fatigue, but more full body fatigue that I now think of as “central nervous system fatigue”. Coping with this is what led me to trying to track my fatigue and various symptoms daily so that I could know when I needed to back off before I ended up just laying on the couch unable to move for hours at a time. This used to happen a lot and at those times, even reading to my three year old was too much.
  • Balance: My balance is pretty bad. I can maybe stand on one foot for 10 seconds. When I have my knee braces on it is a bit better, but still pretty bad.
  • PT recovery: I have trouble recovering from any amount of physical therapy. Especially I think the balance exercises really increases my fatigue. I can do a day or two in a row of my 1.5 hrs of PT, but then I’m very exhausted and in pain for another day or two. Currently I’m getting about two PT sessions a week in, and I’d rather be doing four.
  • Standing: Took me a while to realize that just standing around talking with friends often ends up with a lot of ankle pain later on. I didn’t really understand this until just recently. I think it is a combination of my ankle muscles being weak and my balance being poor which makes me work my ankles more.
  • Running: I can run. Yay! Not far, but mostly because I am out of shape. It hurts at the time though, and it puts me in pain for a few days. The big problem with running is that it increases fatigue and then makes it harder to do my PT exercises. I really want to be running, but it takes so much out of me.
  • Sleep apnea: I was just diagnosed and haven’t really started my machine yet (tonight!), but this is the most exciting diagnosis I’ve ever had because it seems so likely to be solvable. It may also be at the root of a lot of fatigue and recovery problems. 20% of the population has sleep apnea, and a significantly higher percentage of folks with spinal cord injury do (which makes sense given that we have less control of our muscles). Given how easy the test was (basically just take home a device to wear at night) I feel like everyone should get tested.
  • Depression: Ya depression happens a lot. Most of the time it hasn’t been too bad, but sometimes it is completely debilitating. I haven’t started any anti-depressants yet. I’m hoping a lot of this is due to the sleep apnea and lack of sleep (and of course trying to adjust to having paraplegia).

I feel like I keep learning about my symptoms and my body, and I seem to learn something new every few weeks.

Docs, PTs, and Specialization

I am fortunate enough to live near Craig Hospital which is one of the top spinal cord rehab centers in the world. The difference between my docs/PTs at Kaiser (even the ones who know a lot about spinal cord injuries) and Craig has been night and day. I ended up also finding a PT near me who used to work at Craig to go to weekly as well.

Tools and Devices

I’ve been using a bunch of different technologies for tracking and walking.

  • DeRoyal Knee Braces: I’m using these all day long to prevent my knees from hyperextending backwards. Starting with these immediately doubled how far I could walk without ending up in pain. There is a pretty common refrain from folks using wheelchairs that their wheelchair gives them freedom and though my own paraplegia is nowhere near the same, putting on these knee braces gives me an incredible feeling of freedom. I’ve also been doing some of my own modifications to my braces to make them easier and work better for me.
    • Knee sleeves: I found that wearing the braces was causing me a bit of discomfort by the end of the day. Putting on some knee compression bands under them gets rid of that problem almost completely. At first I also tried using Rolyan Cotton Stockinette, but it starts falling apart after about a week so I would constantly need to be making new ones.
    • Marking with silver sharpie where on the straps I am velcroing the cross over straps so I don’t have to spend time trying to figure out how to adjust them each time. I started marking with just some tape and moved to sharpie once I was sure of the location. Simply writing a big L and R on the inside makes getting them on so much faster also.
    • I’m tying a small shoestring from the top flap to the plastic buckle loop to reduce how much the top strap slides down throughout the day. You can see this at the top of my right brace in the picture below.
    • I found that the straps that came with them were not quite the right length so I bought some shorter replacement straps (and then put tape over the logo).

2018-06-15 11.53.31.jpg

  • New Balance 990v4 Running Shoes: I ended up with New Balance running shoes in order to get better stability when walking. They also were wider than the shoes I had been wearing that makes them work better with my orthotics.
  • Orthotics: custom made by my PT at Craig. It was kinda like LEGOs for orthotics, pretty cool.
  • Polar H10 chest strap heart rate monitor: My watch also has a heart rate monitor, but to get good monitoring of heart rate variability (see below) you need a chest strap.
  • Apple Watch: I’m using a bunch of different apps for tracking and building habits. There are a lot of different watches out there, but the breadth of Apple Watch apps is what really helps me:
    • AutoSleep: My only complaint is that I feel like it could have told me six months ago that I should get a sleep apnea test. 🙂
    • Streaks: I’m big into habit building, and this is mostly what I use right now. It is always right in front of my face (on my watch), so lots of reminders throughout the day about what I am trying to work on.
    • Meditate: really nice meditation app that I can start from my watch and is not overly distracting.
    • Apple health for step and distance tracking (also swimming when I can). The HRV data from this is not accurate though.
  • Grip strength tester: I use this as a proxy for how tired my nervous system is (see below).
  • Ankle Stabilizer Brace: Just recently got these. I’m not wearing them all of the time, but if I know I may be standing around a lot or my ankles are in a lot of pain then I’m wearing these. I’m also thinking to try running, hiking, and biking in them to see how much that helps reduce pain. So many experiments…
  • Airex Balance Pad: I use this for a lot of my balance exercises. Makes balancing just a bit harder.
  • BOSU: I haven’t been using this for balance exercises as much recently because it is mostly too hard for me. Need to master things on the Airex first, but expect I will get back to using this too.
  • Other iPhone Apps:
    • Symple: I use this to track symptoms daily and throughout the day
    • EliteHRV: I use this with the chest strap above to measure my heart rate variability
    • YouAte: taking pictures of food
    • MetroTimer: a metronome to help retrain myself to take even steps.

PT Exercises

I’ve gone through a lot of different PT exercises and as I said I’ve worked with a number of different PTs. My current set is focused on getting the exercises really good and solid before moving on to the next set. That was probably one of the biggest mistakes early on with things like balance exercises, I didn’t really have them completely solid before I worked on making them harder.

I’m not going to describe all my exercises, but I will describe how I am rating my ability to do them:

  • Level – I abbreviate this to be something like L1, L2, L3. The levels are arbitrary and a bit dependent on the exercise. For balancing on one foot for 30 seconds for instance the levels are:
    • L1 – eyes open, looking at one point
    • L2 – looking around constantly
    • L3 – eyes closed
    • L4 – eyes on one point, but standing on my Airex
    • L5 – looking around constantly, but standing on my Airex
    • L6 – eyes closed, but standing on my Airex
  • Repetitions – for exercises where I am doing repetitions, this is how many I am doing. Squats for instance.
  • Ability (1, 2, or 3):
    • 1: means that I can’t do something non-stop. For 20 repetitions, I need to stop. For a balance exercise, I fall over.
    • 2: I can do it, but not easily. For balance this means my arms are waving around or that I feel I need to use my shoulders or other muscles I shouldn’t be using.
    • 3: It feels pretty good. Maybe not perfect enough to move on to the next level, but probably will be able to move on in a week or two.
  • Sometimes color coding also – yellow, none, green. Currently I am only using this for balance exercises to indicate whether I am wearing my knee braces and shoes or not.

Ability to do PT exercises doesn’t change very often, so I only record these once a week, and I’m really only looking for improvements over the course of 3-4 weeks at a time. Having some metrics to use and slowly seeing improvement helps with a feeling of having some accomplishment. It will probably also tell me when I have maxed out what I will be able to improve.

Here are 6 weeks of recent tracking.


Daily Fatigue Tracking

I’ve experimented with a number of methods for daily tracking of my fatigue so I would know when to take it easy. I print out spreadsheets for this and tape them to the wall so I can fill them out during the day, and then every week or two transfer it to a spreadsheet on my computer. Here is what I am currently tracking and why:

  • Resting heart rate. Usually goes up above 57 when I am more tired or sick.
  • Hours of sleep (from AutoSleep): varies a lot. Below 4 is bad.
  • Hours of deep sleep (from AutoSleep): if I am not getting deep sleep then my body is not entering the sleep stage where my muscles actually recover
  • Grip Strength: This is used by some elite athletes to track “Central Nervous System Fatigue”. Not great scientific evidence, but for me I find that when my score is low (below 47) then I need to back way off.
  • Heel Raise. 5 point scale: 1 – can’t do it, 2 – very hard, 3 – doable, 4 – have a little bounce, 5 – ready and raring to go
  • Pain on the usual 1-10 scale. At 5 the pain is pretty hard to ignore. If I wake up already at 4, then it is going to be a rough day because my pain almost always increases. This is tricky though because sometimes the pain is due to things unrelated to walking, mostly bowels (that’s a different post).
  • Heart Rate Variability: from the EliteHRV app. Below 47 is usually not very good.
  • Steps (at the end of the day): In general more steps puts me in more pain.
  • Some things I just track in notes, and then summarize later:
    • Fatigue (Y/N)
    • Bad (Y/N)
    • Autonomic Dysreflexia (Y/N) – for me this is triggered by bowel problems

Most of these I record in the morning, except grip strength, pain, and hell raise I do both morning and night. No single metric is enough to say that I am fatigued, but they do give me a sense of whether I need to back off or not. I also track whether I did any PT, walking, running, and notes about my day.

Some recent data:


I’m a pretty big fan of conditional formatting in a spreadsheet to pull out patterns.

Monthly Fatigue/Sensation Tracking

Because my spinal damage is non-traumatic it is possible that it hasn’t fully been fixed and unfortunately the only way to know is if my symptoms start getting worse (at which point they probably won’t get better). So I am doing a bunch of monthly tracking to see if I can find any signals that would indicate that things are getting worse. Most of these are based off of various standard medical questionnaires.

  • Fatigue: I am using the Modified Fatigue Impact Scale (MFIS).
  • Depression: I am also using the PHQ-9 for tracking depression.
  • Sensation changes: I have print outs of the ASIA scale that I am trying to self rate myself with. I basically use a pen cap (perfectly calibrated of course) to poke and rub parts of my body and recored where I am extra sensitive (X), triggers twitching (T), no sensation (O), and numbness (N). Especially looking for differences between each side and changes


I’m also tracking a bunch of bowel and bladder data from month to month, but I’ll leave that for a separate post since I could probably write 3,000 words on that topic too.

Your Turn

OK, so there is everything I am tracking around sensation and movement so far. I’d be really interested in anything others have found useful. This is already a lot, but since I keep learning new things, I assume there is a lot more to think about that I am missing.

Huh, Seven Years. Look at that…

It turns out that seven years ago today I started my trial project with Automattic. I know this because that is when I signed up for user id 23314024 on WordPress.com

Screen Shot 2018-05-11 at 6.25.55 PM.png

My official Automatticversary – as we call the day you became a full time employee – is not until mid-July, but this notification inspired me to be a bit nostalgic. It also happens to fall on the week of my 40th birthday.

My trial project was building a prototype Elasticsearch system (version 0.16 IIRC) so we could search across our internal p2s (disclaimer: no longer what p2 looks like). At the time there were 291k posts and comments across our 89 p2s and MySQL search was no longer working well. There are now almost 2.8m posts and comments across 639 p2s and Elasticsearch is continuing to do pretty well.

We now have six Elasticsearch clusters the largest of which had 6,782,578,164 posts in 2625 shards a few seconds ago. Those clusters have about 850 separate indices that are powering over 66 different use cases. Pretty exciting and humbling how much the company’s usage has grown.

If you are looking to get in on the Search Wrangling fun, then we are hiring. We have a ton of challenging search and relevancy problems to tackle.


WordPress and Democratizing Algorithms

In discussing how the newly released Jetpack Search fits with Core WordPress search I veered off into discussing algorithms in general. That generated some healthy discussion on the WPTavern Podcast. I wanted to expand on what I was saying a bit about democratizing algorithms. This is very much related to what JJJ called “race car technology makes its way down to everybody” on the podcast.

I think some of the negative reaction is a personal preference where many readers loudly prefer reverse chronological because it is easy to understand and they feel in control. I think that is a very common opinion among WordPress developers. I’ve had that discussion many, many times. I also think that user control is important.

However, it is very clear that ON AVERAGE having some algorithm filter or reorder the content is much more engaging. It boosts visitor engagement.

Let’s look at some top examples: Twitter, The New York Times, The Washington Post, and Amazon (30% of humans use Facebook, so let’s stipulate that algorithmic is working pretty well for them).


Many people are loudly complaining about Twitter not being completely reverse chronological. However, as a product and service, they just had their first profitable quarter ever. And it wasn’t because they have more users. Monthly active users is flat. But daily active users increased 12% year-over-year. They have steadily made their product more engaging to the average user. So while yes many people dislike the algorithmic changes they are making, they also seem to be working.

New York Times and Washington Post

nytimes-top.pngwashpost.pngBeyond even human “algorithms” that order which articles are on the front page, it is very common to have algorithmic sections of the front page of major news sites. Sometimes it is “most emailed stories”, or sometimes it is the most viewed. The current NYT’s home page has a section which looks like the top posts from across the past few days in a nice scrollable display. Maybe they are hand picked, but I bet there is data influencing them. The Washington Post has “Most Read” and “Live Discussions” sections.

Even on this blog I’m using Jetpack’s top viewed posts widget and Jetpack related posts. Both are things that WordPress Core can’t do.


Algorithms are everywhere on Amazon. Let’s look at the dog food I buy. The reviews are clearly sorted by what will get me to buy it.

dog-food.pngHow could I not buy it!

It is about Engagement!

None of these examples are easy (or cheap) to deploy right now. It is still pretty hard to build an engaging website. Maybe building an engaging website is plugin territory. That sounds great for Jetpack which has more and more of these features. I’m not sure it is great for the Web though.

This even comes up with simple websites. Gutenberg opens up very interesting questions about how blocks should be organized on a web page. Chris Lema mentioned this when discussing the future of web publishing:

The future can’t continue to be a unidirectional dynamic where someone in marketing determines the best articulation of their message in a single-focused and static design.

The future of publishing is that different people can get different content depending on their behavior, demographics, interest and more.

I think framing it as personally preferring algorithmic vs chronological is really the wrong way to think about it. The question is, if you could flip a switch and get your visitors to spend 20% more time on your website, then would you?

Right now, only big expensive web sites can toggle that switch. Many of them are effectively monopolies. Let’s democratize algorithms so any website can choose to build a highly engaging user experience.

An Aside

While we can and should talk about filter bubbles and the impact that these algorithms have on the world, a world where only monopolistic tech giants can deploy these algorithms is not one where publishing is democratic.


Combining Search Scores: Winning and Failing

Trey Jones at Wikimedia Foundation published some very interesting notes up about how to think about combining scores for search ranking (particularly Elasticsearch). I like this insight a lot:

addition is looking for ways to win, multiplication is looking for ways to fail

This is pretty interesting to me when thinking about how I chose to implement the ranking for the WordPress.org plugin search. Applying this insight to the way I combined signals in that ranking function comes up with a couple of interesting observations:

  • The text matching features (phrases, title matches, etc) are looking for ways to win and boost the score. This was a pretty explicit goal of mine, but also partly driven by decoupling the matching of text from boosting on text.
  • All of the other signals are looking for reasons to fail. Not updating the plugin, not testing it on latest WordPress, not resolving support threads, etc. There is some boosting also, but we do a lot to lower scores which is maybe related to some of the exact matching problems I am still looking at (especially after result number 10).

I’m not sure this is either good or bad, just an interesting model for thinking about it and something I need to think about some more. This somewhat matches the intuition that led me to separate out matching text from boosting text with individual features.

I also need to think more about whether I am using the right operations for weighting different scores. There’s a lot of great thoughts in these notes and Trey has a bunch of other notes that look interesting also.

Also it reminds me how great it is to have notes published for others to look at.


Senator Bennet, your email address doesn’t work.

I emailed my Senator a few weeks ago to oppose Gorsuch being appointed to the Supreme Court. His response seems to indicate he thinks Trump nominees are still worthy of consideration. That’s an absurd stance since they have been found to be lying under oath during the confirmation process.

Here is Senator Bennet’s response:

Dear Gregory:

Thank you for contacting me regarding the current vacancy on the Supreme Court.

On January 31, 2017, President Trump nominated Judge Neil Gorsuch to serve on the Supreme Court. He has served on the 10th Circuit Court of Appeals since 2006.  Gorsuch clerked for Judge David Sentelle on the U.S. Court of Appeals for the D.C. Circuit and for Supreme Court Justices Byron White and Anthony Kennedy. He also served as the Principal Deputy to the Associate Attorney General at the U.S. Department of Justice.

I take seriously the Senate’s constitutional responsibility to thoroughly vet Judge Gorsuch. I intend to review his record carefully in the coming weeks. Rest assured, I will keep your thoughts and concerns in mind throughout the confirmation process.

I value the input of fellow Coloradans in considering the wide variety of important issues and legislative initiatives that come before the Senate. I hope you will continue to inform me of your thoughts and concerns.

For more information about my priorities as a U.S. Senator, I invite you to visit my website at http://bennet.senate.gov/. Again, thank you for contacting me.

Below is my response

Senator, thanks for your response.

He has served on the 10th Circuit Court of Appeals since 2006.  Gorsuch clerked for Judge David Sentelle on the U.S. Court of Appeals for the D.C. Circuit and for Supreme Court Justices Byron White and Anthony Kennedy. He also served as the Principal Deputy to the Associate Attorney General at the U.S. Department of Justice.
None of this matters. You should be working to delay each and every nomination by Trump because the longer we delay them the less damage he can cause.
There should be no more confirmations until Sessions has resigned for lying under oath during his confirmation process. How can you trust anything that any Trump nominee has said in any hearing until it is clear that they take not lying to Congress seriously?

Alas… it seems he doesn’t want to accept email responses…
Your message was sent to a non-monitored mailbox and has not been reviewed. If you would like to contact Senator Michael Bennet please visit his website at http://bennet.senate.gov/contact and fill out the webform for a prompt response. Thank you.
So much for dialog… Hey look I have a blog…

Top Five Posts from 2016

Most of this blog’s 40k visitors a year are looking at the epic Elasticsearch posts that I wrote years ago. For the most part they seem to still be relevant to people even if they are somewhat outdated. Here are my top posts with some commentary about each of them.


1: Elasticsearch: Five Things I was Doing Wrong

79% of my traffic comes from search engines, and almost 50% of all traffic goes to this one post. It’s actually kinda crazy that such a simple post gets so much of my traffic. I blame the clickbait headline. I have a bunch of long winded epic posts and what I should probably be writing is these small tidbits as they come up.

2: Three Principles for Multilingal Indexing in Elasticsearch

This is my all time favorite post. After 2.x and the removal of being able to specify an analyzer in a query it has become a bit outdated, but the overall concepts are still good. I love all the comments this post has generated. I’ve learned so much from this post and the discussions that it generated. We’ve accomplished a lot the past year to adjust our multi-lingual indexing (deployed edgengrams into an A/B test yesterday) and I’m hoping to write up what my latest thinking is soon.

3 and 4: Scaling Elasticsearch Series

The first two parts of this three part series are my third and fourth most popular posts. The indexing post is almost twice as popular as the intro and querying posts. Although these posts are almost three years old now they still describe pretty well how we scale most of our queries. Most of the reason why these posts haven’t been updated is because the methods they describe have worked really well for us.

The original post talks about having 600 million posts in the index and 23m queries a day. We now have 4.3 billion posts and do about 45m queries a day. That’s some good scaling.

Only over the past year have we started to see some problems slowly develop with our global cluster scaling. Currently the cluster runs fine for about a month or so and then heap usage creeps upwards until it starts to cause problems. The solution is just to do rolling restart of the cluster. Not pretty, but it works. Here’s what our average heap usage looks like broken down by data center for the past 30 days.

Screen Shot 2017-01-06 at 9.28.48 AM.png

We think a lot of these are just memory management bugs in the old Elasticsearch version we have been running for years and are hopeful that as we transition to 2.x many of them will be resolved. The other option is just to add more servers which we haven’t done in a few years. Our typical load is not very high though until we reach the point of running out of heap so I haven’t felt very justified in ordering more servers for this cluster yet.

One high point of this cluster is it taught us how to run a multi data center cluster. Every cluster we deploy now is multi-data center and we have successfully survived cases where an entire data center goes down. Currently we are in three data centers spread across the US. It’s likely that in 2017 we will start trying to run intercontinental Elasticsearch clusters (Europe and the US). Should be exciting.

5: Managing Elasticsearch Cluster Restart Time

This post describes how we manage long restart times. 2.x is a bit faster in this regard, but still takes a while to synchronize, so this is still relevant to managing a production ES cluster.