Visualizing the tides of San Francisco Bay

Just for fun, here is every high and low tide for San Francisco Bay in 2016. Red dots are the high tides; blue dots are the low tides. (click to enlarge)

sf high and low tides
data source

I found it interesting that the highest low tide of the year is lower than the lowest high tide of the year. There’s a certain narrow range of water levels — about 3.1-3.4 feet — which sees neither high tides nor low tides. Perhaps there’s some obvious reason for this, known to oceanographers, but it was news to me! I’m also curious if the non-overlap of high and low tides is true everywhere?

Any tide gurus out there?

The warmest winter on record in San Francisco Bay

Here in San Francisco, the common sauna wisdom is that we just experienced one of the warmest winters in recent memory. The Dolphin Club’s Polar Bear Challenge was hardly challenging, and the South End’s “Dreaded 9th” of February swim was hardly dreaded.

Just how warm was it, though? I crunched the numbers from the NDBC, because, well, why not.

Here we see the last 15+ months of data from the Crissy Field station (FTPC1) inside San Francisco Bay, plotted in solid black. The dashed green, red, and blue lines show the long-term average, maxima, and minima for each day of the year, summarized over the eight years of available data from that station.

san francisco bay water temp

From July 2014 until just the past few days (early April 2015), Bay waters have been hovering 2-3 degrees (F) above the all-time highs (going back to 2006), and about 5 degrees above the long-term averages.

Eight years isn’t much data, unfortunately. Can we do better?

A bit: Lightstation 46026 – about two-thirds of the way out to the Farallones – has data going back to 1982. Almost 33 years of data! This is colder, hairier water than inside the Bay, but it’s as close as we’re going to get.

Here’s a similar chart for this much longer-term data set. Unfortunately, the water temp sensor at Station 46026 was out of commission for the first eight months of 2014, so we can compare “recent data” starting from late August.

san francisco bay water temp

The story is similar, though: This was one of the warmest winters on record. Through most of December, we were ~4 degrees above the all-time highs going back over 30 years.

Indeed, when I took the average sea temps for each full winter season (December 1 through February 28), I found the following five warmest winters since 1982-83:

  • 2014-15     57.3 (F)
  • 2004-05     55.0
  • 2002-03     54.7
  • 1982-83     54.6
  • 1983-84     54.3

Only time will tell if the recent drop in Bay temps is a momentary aberration, or a longer-term regression to the mean.

No doubt, this summer’s Farallon swimmers will be watching this closely. 2014 saw the first two Farallon solos since 1967 — Craig Lenning and Joe Locke. And from a water temp perspective, 2014 was probably the best year to swim the Farallons in over a generation. We won’t always be so fortunate.

Data crunched with R and plotted with ggplot2.

Information Wants to be Free: Sandettie Lightship and the English Channel

(tl; dr — 10 years of English Channel weather data, in a single CSV file. And some fun charts.)

Weather can turn on a dime in the English Channel, and the dreams (and finances) of English Channel swimmers often turn on the weather.

sandettie lightship
Location of Sandettie Lightship in the English Channel

The most important source of information about that weather is a 156-foot lightvessel called Sandettie, which serves as both a floating lighthouse and a weather station. Here’s a nice photo.

Sandettie collects a variety of important meteorological data – air and sea temperatures, wind speed and direction, wave height and period, humidity, and barometric pressure. These data are then fed back to the UK Met Office, who publish the most recent 24 hours’ of observations on their website.

Anything before the last 24 hours are what the Met Office call “chargeable data” — at the rate of £6800 per 10 years, per two elements (e.g., air temp & sea temp). According to the today’s exchange rate, that converts to no less than $11,575 USD.

LOL! (And yes, I actually requested a quote from the Met Office.)

Just sayin’: In the US, quality-controlled meteorological data are available from NOAA’s National Data Buoy Center — for free.


Data on historical air and sea temperatures (going back to 2004) are available from the Channel Swimming & Piloting Federation, in the form of interactive charts. Thanks to CS&PF tech whiz Boris Mavra, these charts are automatically updated from the Met Office’s recent observations table.

The CS&PF charts are pretty slick, but personally I’d rather have the raw data to play around with. Not just air & sea temps, but also the wind and the waves. The raw data allow one compute (among other things) summary statistics – e.g., What’s the typical sea temp in the third week of August (averaged across many years)?

But clearly, my curiosity isn’t worth $20,000+ (extrapolating the Met Office’s rate for two elements). So what am I to do?

Other sources of weather data include commercial (non-government) weather services and websites – you can probably think of a few. I managed to find one such website with what appears to be more than 10 years of Sandettie data (going back to June 19, 2004 — same start date as the CS&PF data). All freely and publicly accessible.

Unfortunately, these data are formatted rather inconveniently – one day at a time, in HTML tables. Ugh! You could sit there all day, pointing, clicking, copying, and pasting into Excel, for each one of the 3655 days between June 19, 2004 and today. That would be a ridiculous way to spend a day, but it’s not inconceivable.

Or…you could program a computer to do it for you. So, harnessing the powerful data-munging capabilities of R, that’s what I did. Here’s the code, if you’re the sort of person who’s interested in such things (spirit of open-source, etc.):

The result, after a merge and a de-dup? A tidy, comma-delimited 81,901-row data-set with hourly observations on eight variables:

  1. air temperature (degrees Celsius)
  2. sea temperature (degrees Celsius)
  3. humidity (percent)
  4. wind direction (16-level factor: N, NNW, NW, WNW, W, etc.)
  5. wind speed (knots)
  6. wave period (seconds)
  7. wave height (meters)
  8. barometric pressure (hPa)

Here’s the compressed .CSV for your downloading pleasure:

I’ve inspected the data for any gross integrity issues, but have made no additional effort (thus far) to “clean” it of anomalies. As the CS&PF note regarding their own Sandettie data-set:

Data quality: it is easy to see that there are glitches in the way station sensors work or the way they report the measurements. We are planning to clean the records in the near future, but for now we rely on readers’ intelligence in interpreting the feeds. We all know North Sea does not freeze in one hour and 100 mph winds in the middle of the summer are very unlikely!

There are definitely some anomalies (see charts below), but they appear fairly normally distributed. So, any subsequent “cleaning” should be reasonably straightforward.

These charts aren’t meant to be taken too seriously (they each required just a single line of R code) — just as a first step in exploring and validating an interesting data-set.

Click any to enlarge:


Actually, relatively few anomalies, considering this represents nearly 82,000 observations!

atmp_sm wspd_sm

The next chart shows the same data as above, but I’ve zoomed-in the Y-axis to eliminate the most extreme anomalies.

wspdZ_sm wvht_sm

Again, the next chart shows the same data as above, just with a zoomed-in Y-axis.

wvhtZ_sm pres_sm


Many thanks to Hadley Wickham for his wonderful ggplot2 package for R, which I used to create these charts.

Important Note: Technically, because I did not obtain these data directly from the UK Met Office, I can make absolutely no guarantees about their integrity or authenticity. However, I will say that subjectively speaking, they “look right.”

Another Important Note: If these data are authentic, then they are considered to contain public sector information licensed under the UK Open Government License v1.0.

A Final Important Note: As far as I know, the extraction (“scraping”) of the data from the third-party weather service did not violate its Terms of Service, which explicitly permit using the data for personal, non-commercial purposes. And I define this blog post as a personal, non-commercial purpose.

Most popular threads in the history of the Marathon Swimmers Forum

Since launching in March 2012, the Marathon Swimmers Forum has grown to 935 verified members who have contributed 13,195 comments in 796 discussion threads.

Of those 796 threads, the most popular by far (as measured by page-views) was the discussion of Diana Nyad’s purported swim from Cuba to Florida in September 2013.

The top 25 threads in the history of the Forum are as follows. Relative page views for each thread are shown (indexed to the Nyad thread = 100).

  1. 110 miles, 53 hours: Questions for Diana Nyad [100]
  2. Best Goggles for Open Water Swimming and Triathlon [34.6]
  3. 10km training schedule – 3 months out [11.1]
  4. Waterproof MP3 players for swimming [9.4]
  5. Diana Nyad’s Directional “Streamer” [7.9]
  7. You might be an open water swimmer if … [6.1]
  8. Recommended Products [5.8]
  9. The Animal Set Thread [5.8]
  10. Swim Videos [5.2]
  11. training for a 5km swim [5.2]
  12. Manhattan Island Marathon Swim 2013 [5]
  13. What is an appropriate weekly mileage for completing marathon swims? [4.6]
  14. Sunscreen?! [4.5]
  15. Warming up after a cold water swim [4.3]
  16. Driven [4.2]
  17. Open Water Blogs [4.2]
  18. Diana Nyad’s epic swim [4]
  19. What type of Garmin is recommended for tracking distance and time of open water swims? [4]
  20. Questions about Feeding [4]
  21. Do marathon swimmers breathe every stroke? [3.8]
  22. Craig Lenning – Farallon Swim [3.6]
  23. What’s your next swim? [3.4]
  24. Should rash guards / swim shirts / full body suits (not wetsuits) be permitted? [3.4]
  25. Who Earns the Title ‘Channel Swimmer’? [3.4]

A better SWOLF formula

SWOLF (“swim golf”) is a drill that measures swimming efficiency. A SWOLF score is your time (in seconds) on one lap of the pool, added to the number of strokes you took. Lower scores = Higher efficiency. SWOLF is a fuzzy, indirect measure of efficiency, because stroke count doesn’t necessarily reflect effort. In my view, the most precise definition of SWOLF is that it identifies the most efficient stroke count for a given level of effort.

I originally wrote about SWOLF in April 2012, and the post has become – by a wide margin – the most widely-read in the history of this blog. In a subsequent post a month later – “Stroke Count Games” – I described how SWOLF doesn’t quite capture the most efficient stroke count. At least for me, using stroke cycles (number of strokes divided by two) produces better results.

I wondered if this was true for other swimmers, so I asked any interested readers to send me their own data, using a test set of 8×100. Three readers sent me their results. In short, my suspicions were confirmed: Strokes cycles produces better results than stroke count. In essence, the original SWOLF formula seems to over-weight stroke length in its measure of efficiency (and thus under-weight speed).

Reader #1 is a 6-foot 2-inch (188 cm) male in his late-20’s with an ape index of 1.07 (arm-span greater than height). He did not swim competitively at the high school or college level. His recent best times include 25:21 for the 1650-yd Freestyle and 3 hours, 18 minutes for a 10km open-water swim. His typical open-water stroke rate at marathon pace is approximately 50 strokes per minute.

Here are his results for the test set of 8×100:

SPL m:ss SWOLF SWOLF-improved
10 1:50 150 130
11 1:40 144 122
12 1:30 138 115
13 1:23 135 109
14 1:16 132 104
15 1:12 132 102
16 1:13 137 105
17 1:15 143 109
18 1:17 149 113

Reader #1 pegged his “natural” stroke count per 25 yards at 15-17. According to traditional SWOLF, he was most efficient at 14-15 SPL, followed by 13. According to SWOLF-improved, he was most efficient at 15 SPL, closely followed by 14 and 16 SPL. SWOLF-improved seems slightly more accurate in this case. In Reader #1’s own words:

15-17 feels natural.  At 14 I could already notice some laboring.  Anything at 13 or lower, inertia was a huge factor.

I think that’s why the curve is much steeper on the lower stroke side.  It might have said 13 was more efficient than 17, but no way I’d want to swim more than a 100 at 13.  17 – no problem.

Reader #2 is a 5-foot 7-inch (170 cm) female in her late 20’s with an ape index of 1.0 (arm-span equal to height). She swam competitively in both high school and college, and is a Triple Crown marathon swimmer. Her recent best times in the pool include 19:15 for the 1650-yd Freestyle. Her typical open-water stroke rate at marathon pace is 70 strokes per minute.

Reader #2 insists her data include the caveat that she did a big training swim (21km) the previous day 😉

SPL m:ss SWOLF SWOLF-improved
15 1:20 140 110
16 1:18 142 110
17 1:15 143 109
18 1:12 144 108
19 1:10 146 108
20 1:08 148 108
21 1:08 152 110
22 1:10 158 114

Reader #2 puts her “natural” stroke count per 25 yards at 18-20, depending on pace. SWOLF-improved agrees. Traditional SWOLF, on the other hand, under-estimates her most efficient stroke count.

Reader #3 is a 5-foot 6-inch female in her early 20’s. She swam competitively in both high school and college, and is a national-caliber distance swimmer. Her recent best pool times include 17:09 for the 1650-yd Freestyle. Her typical open-water stroke rate is approximately 80 strokes per minute.

SPL m:ss SWOLF SWOLF-improved
12 1:14 122 98
13 1:10 122 96
14 1:08 124 96
15 1:04.5 124.5 94.5
16 1:02 126 94
17 1:00 128 94
18 0:59.5 131.5 95.5
19 0:59 135 97

Reader #3 puts her “natural” stroke count per 25 yards at 16-17. SWOLF-improved agrees. Traditional SWOLF, once again, underestimates the most efficient stroke count.

For comparison, here are my results from the 8×100 set, as reported previously:

SPL m:ss SWOLF SWOLF-improved
9 1:20 116 98
10 1:14 114 94
11 1:10 114 92
12 1:07 115 91
13 1:05 117 91
14 1:02 118 90
15 1:00 120 90
16 1:01 121 93

In sum, you may find that using stroke cycles instead of stroke count produces more useful SWOLF results. If you own a Swimsense, then you’re golden – it already uses stroke cycles for its SWOLF calculation. I don’t own a Garmin Swim watch, but from what I’ve read online it seems to use the traditional formula.

Marathon Swimming Rules Survey: Results and Analysis

Marathon swimmers talk a lot about rules – what should and shouldn’t be allowed during a swim – but as far as I know, there has never been any systematic study of what marathon swimmers actually think, as a matter of public opinion.

Perhaps most would agree that goggles are OK, and fins are verboten… but what about swim streamers and stinger suits? Or drafting off the escort boat? If you only read blogs and forums, you might assume the most vocal opinions represent the majority. But do they really?

Earlier this month the SBCSA launched a survey to find out. Over 25 days, we received 175 responses from around the world.

First, a Summary of Findings (TL/DR). Click any of the following links to skip directly to the relevant section.

I. We received responses from a representative sample of marathon swimmers – current, former, and aspiring.

II(a). Marathon swimmers agree on basic channel-rules attire: traditional porous textile swimsuit (including jammers), goggles, one latex or silicone cap, ear plugs, and nose clips.

II(b). Marathon swimmers agree that substances or devices that protect the swimmer against dangerous marine life (e.g., sharks & jellyfish) – but unambiguously do not enhance performance – are acceptable.

II(c). Marathon swimmers agree that devices or substances that unambiguously enhance speed, buoyancy, or heat retention should NOT be allowed on marathon swims.

III. Controversial items include stinger suits, swim streamers, bubble caps, and shark divers.

IV. The more marathon swimming experience a person has, the more likely she/he is to embrace a minimalist approach to swim aids.

I. We received responses from a representative sample of marathon swimmers – current, former, and aspiring.

To argue that this survey accurately represents the opinion of the marathon swimming community, we must show that the 175 respondents are a representative sample of the community. We can do this in a few ways.

A. Geography

Of the 175 respondents, 71% live in North America, 19% live in the United Kingdom or Ireland, 5% live in Australia or New Zealand, and the remaining 5% live elsewhere.

As a baseline for comparison, here’s how those numbers compare to the unique visitors to the Marathon Swimmers Forum in February:

Geographical distribution: Survey respondents vs. Marathon Swimmers Forum visitors

Another baseline for comparison? The Triple Crown list: as of 2012, 76% are from North America, 10% from the UK+Ireland, 4% from Australia+NZ, 4% from continental Europe, and the rest from elsewhere.

In sum, the survey sample has a lot of North Americans – but then, so does the global marathon swimming community generally.

B. Gender

Gender distribution of survey respondents

What about the Triple Crown list? Exactly 60% men, 40% women. Pretty darned close.

C. Self-identification as a marathon swimmer

We asked respondents what they “identify most closely as.” Although we didn’t forbid non-marathon swimmers from taking the survey, we promoted and targeted it primarily at marathon swimmers, because that’s what our primary interest was: What do marathon swimmers think?

According to the data, 87% of respondents identified as either a current, former, or aspiring marathon swimmer.

Self-identification of survey respondents
Self-identification of survey respondents

D. Marathon swimming experience

We asked survey respondents about their specific experience in marathon swimming (and other endurance sports). We found that:

  1. 90% of survey respondents have swum at least 10km in open water.
  2. More than half have swum at least 25km in open water.
  3. Almost a third have swum the English Channel.
Accomplishments of survey respondents.
Marathon swimming experience of survey respondents.

Interesting sub-finding: Marathon swimmers are not as challenged on terra firma as the stereotypes might suggest. Almost half of respondents have done an Olympic-distance triathlon (or longer), and 30% have run a marathon. In comparison, Runners World estimates the percentage of the U.S. population who have run a marathon at 0.5% (ref).

II. Marathon swimmers largely agree on what should (and should not) be used in their sport.

Now to the meat of the study. What do marathon swimmers agree on?

Some critics and swim-aid proponents would have you believe the marathon swimming community can’t agree on what their own rules are. The implicit argument is typically: “Therefore, we might as well just let people use anything they want.”

Actually, the marathon swimming community agrees on quite a lot.

A. The marathon swimming community agrees on basic channel-rules attire: traditional porous textile swimsuit (including jammers), goggles, one latex or silicone cap, ear plugs, and nose clips.

B. The marathon swimming community agrees that substances or devices that protect the swimmer against dangerous marine life (e.g., sharks & jellyfish) – but unambiguously do not enhance performance – are acceptable.

More than 75% of survey respondents agreed that the following items are acceptable:

Percent of respondents who agree that item should be allowed on marathon swims
Percent of respondents who think item SHOULD be allowed on marathon swims

C. The marathon swimming community agrees that devices or substances that unambiguously enhance speed, buoyancy, or heat retention should NOT be allowed on marathon swims.

(Including drafting off the escort boat, which is allowed in the English Channel.)

More than 75% of survey respondents agreed that the following items are NOT acceptable:

Percent of respondents who agree that item SHOULD NOT be allowed on marathon swims
Percent of respondents who agree that item SHOULD NOT be allowed on marathon swims

D. More moderate consensus exists on the following:

Percent of respondents who think item should be allowed

Some thoughts on why there is less consensus on these items:

  1. Using boat to shield from wind & waves – improves performance, but is already widely allowed, and it’s unclear how a prohibition could be enforced.
  2. Exiting water for safety reasons – allowed in MIMS and Cook Strait, but not elsewhere.
  3. Topical substance that retains body heat – does such a substance even exist? Perhaps a confusing question.
  4. Multiple caps – allowed by FINA, minimally performance enhancing.
  5. Shark sharpshooter – not performance enhancing, but harmful to sharks and thus morally problematic.
  6. Topical substance that warms the body – does such a substance exist? Confusing question.

III. Controversial items: stinger suits, swim streamers, bubble caps, and shark divers.

A. Shark divers. 59/41 (for/against).

B. Bubble caps. 43/57 (for/against).

C. Swim streamers. 46/54 (for/against).

D. Stinger suits. Tie – 50/50. 

(If you must know, the stinger suit vote was 84-yes, 83-no, with 8 no answers.)

My view: if an item is controversial, it cannot be considered “approved by the sport of ocean swimming.” At best, it might be considered a “local exception” to a more universal set of rules – for example, the use of streamers in Japan.

If an item is controversial, it is in some way approaching a line in the sand. In marathon swimming, if you’re flirting with this line – trying to find loopholes for some extra edge – quite simply, you’re doing it wrong.

Some stinger suit proponents claim that these enhanced-coverage suits are merely protective, not performance-enhancing – and that therefore they should be allowed on marathon swims.

Personally, I’m not sure about this claim. Couldn’t someone easily produce a stinger suit that is performance enhancing? Would we then have to define new rules about what is and is not a performance enhancing stinger suit? Could I put on my old full-body Blueseventy Nero tech suit and call it a “stinger suit”?

IV. The more marathon swimming experience a person has, the more likely she/he is to embrace a minimalist approach to swim aids.

The data presented so far represent the “collective” opinion of the marathon swimming community. However, within that collective, there is actually quite a diversity of opinions among individuals. For example, one person might think a streamer is OK but a stinger suit is not OK; while another person might think a streamer is not OK while a stinger suit is fine.

This diversity of opinions in the survey sample ranged from:

  • One extremely purist/minimalist individual who would only allow a standard cap, goggles, grease, sunblock, boat navigation, limited pace swimming, caffeine, anti-inflammatories, and touch starts. This person would prohibit everything else.
  • One extremely liberal-minded individual who would prohibit nothing – i.e., everything should be allowed (a troll, perhaps?).

For each survey respondent, I summed the total number of items the individual would allow – as an ideology index. So the minimalist respondent I mentioned above would get a 9 on the ideology index, while the everything-is-allowed respondent/troll would get a 48.

Here’s how the respondents were distributed according to ideology:

Histogram showing number of respondents grouped by how many items they would allow.
Histogram showing number of respondents grouped by how many items they would allow. Each number on the X-axis represents a “basket” of 5. So, the people in the ’25’ basket are those who would allow between 21 and 25 items, out of a possible 48.

One interesting question is: Why do some people prefer a minimalist approach, while others embrace technology and swim aids?

We would need a much longer survey to tease out the various reasons, but even in this brief survey there is a clear pattern:

The more marathon swimming experience a person has, the more likely she/he is to embrace a minimalist approach to swim aids.

The following chart shows the average “ideology index” score (out of 48) for four groups:

  1. People who have never done a marathon swim (27 of 175 total respondents)
  2. People who have done a 10km open-water swim but not a 25km (56 of 175)
  3. People who have done at least a 25km swim or one of the Triple Crown swims (57 of 175)
  4. People who have done two or three of the Triple Crown swims (35 of 175)
Average ideology score, depending on marathon swimming experience
Average ideology score, depending on marathon swimming experience

The same pattern emerges when we look at people’s opinions on just a single item, for example, the controversial stinger suit.

Percent of respondents who think stinger suits should be allowed, according to marathon swimming experience
Percent of respondents who think stinger suits should be allowed, according to marathon swimming experience

Obviously there’s much more we could get into with this data, but for now this report is quite long enough already. And I think I covered the big points. If readers are interested, I will do a follow-up post with additional summary data and analyses, as requested — an “appendix” of sorts. Let me know what you want to know.

For reference, here are screenshots of the original survey (click to enlarge):

Related External Posts

Google search trends and open water swimming

Google has a fun tool that lets you visualize trends in search queries submitted by its users. Google is often the first place people go to find out more about a given topic, so it’s a powerful measure of the public’s “interest” in that topic. Below are a few Google Trends graphs related to open water swimming.

Is open water swimming “growing”? 

search term: “open water swimming”

Some observations:

  • Interest in open water swimming is highly cyclical, with summer peaks and winter troughs. (Obviously.)
  • Two big “spikes” corresponding to the Olympic Games in 2008 and 2012.
  • Aside from the seasonal cycles and Olympic spikes, the peaks and troughs do seem to rising slightly over time.

What about two sub-genres of open water swimming: marathon swimming and triathlon swimming?
search terms: “marathon swim” (blue) vs. “triathlon swim”

As expected, triathlon swimming is consistently bigger than marathon swimming. One exception: the surge of interest associated with the London Olympic 10K marathon swim.

What about the Triple Crown events: English Channel, Catalina Channel, and the Manhattan Island Marathon Swim?

search terms: “english channel” swim (blue) vs. “catalina channel” swim (red) vs. “manhattan island marathon swim”

Catalina and MIMS hardly register a blip, compared to the interest in the English Channel.

The two pre-eminent schools of adult swimming development: Total Immersion and Swim Smooth:

search terms: “total immersion” swimming (blue) vs. “swim smooth”

TI is the longtime incumbent. But in late 2012, for the first time since Google has tracked these data, Swim Smooth has overtaken TI in search queries. Perhaps, people are realizing Swim Smooth offers a better product.

Three of the most famous marathon swimmers: Diana Nyad, Lynne Cox, and Penny Palfrey:

search terms: “diana nyad” (blue) vs. “lynne cox” (red) vs. “penny palfrey”

For whatever reason, people are fascinated by Diana Nyad. Penny Palfrey’s more successful, more legitimate Cuba-to-Florida attempt this past summer hardly registered a blip in comparison.

As big as the Great Nyad seems in our little sport, how does she compare to stars in the pool, such as Michael Phelps?

search terms: “diana nyad” vs. “michael phelps” (red)

And that’s why Phelps is worth $45 million, while marathon swimmers beg for sponsors.

Choose your sugar:

search terms: “hammer nutrition” (blue) vs. “gu energy” (red) vs. cytomax

Hammer overtook Cytomax in 2008 and hasn’t looked back. GU Energy Labs also seems to be making consistent progress in the marketplace.

Other ideas for search query comparisons? Try it yourself.