A Deep Dive into Debugging Cypress, iframes, & Runtime Code Modification

As software engineers, a large portion of our time is spent researching, reproducing, and fixing bugs. Despite our best intentions, careful code craftsmanship, and thorough testing, these software defects will still discreetly and insidiously make their way into our applications. This can lead to degraded user experiences, application downtime, and potentially a loss of revenue for our businesses. Being able to effectively discover, track down, and resolve bugs is one of the most important skills in our toolboxes.

Recently, I found myself tackling one of the most difficult, bewildering, and intricate bugs that I have ever faced thus far in my career. It took all of the debugging techniques I have learned as a developer (and then some!) to solve and fix it. I'm hoping that you may find some value or learn something new in this recount of the steps I took, tools I used, and misdirections I encountered in my clash with this bug.

The Payment Form Iframe

Let's set the scene. I was working on a client project where we were building a payment/checkout form to be embedded across a suite of UI applications as an iframe. As part of our efforts to create a secure experience for our users, we were using a third-party payment processor/gateway service called Spreedly. As part of Spreedly's offering, a JavaScript library is available that provides functionality to tokenize and store payment methods such as bank accounts and credit cards. In addition, the library initializes and embeds two Spreedly-managed input fields, one for the user's credit card number and the other for the CVV, in a series of iframes.

Now, you may have already noticed, but we were working in an iframe inception. We were providing our own iframe to clients which then, in turn, had multiple Spreedly iframes embedded within that.

To make this easier to conceptualize, let's examine what this form might actually look like and where boundaries of the various iframes are.

Spreedly injects a parent iframe within our own form's iframe, which then subsequently renders two children iframes for each of the Spreedly-managed fields.

Now that's a lot of iframes, but you may be asking yourself "Can we go deeper?" The answer is a resounding yes! We also introduce yet another iframe during our end-to-end testing with Cypress. When Cypress Test Runner executes our tests, it loads our application into its own iframe and context. This is where we first encountered the bug.

Bug Ticket #5629

Little did I know of the journey I would go on when taking on bug ticket #5629. The gist of this bug was that the Spreedly-managed credit card number and CVV input fields were failing to render and display in Cypress and as a consequence, were causing cascading failures in our tests. However, when loading and running our application in its normal context in the browser, everything worked fine and we could not reproduce the bug outside of Cypress despite our best efforts. To make matters worse, the error event handler that was built into the Spreedly library wasn't logging any useful information.

Surely, though, this would be a simple fix and probably revolved around something like environment variables or environment-specific configurations. I dove into various parts of our codebase looking for these sorts of culprits only to come up empty handed. There were simply no environment-specific configurations or variables that would cause deviations in behavior for this part of our application.

Was it possible that the JavaScript library provided by Spreedly was doing something weird based on the fact that it was running within Cypress' iframe? This seemed unlikely, especially since we had our iframe loading and working within Storybook, which like Cypress, rendered our application and components in its own iframe. Was there a bug in Cypress that was causing this defect? This also seemed slightly improbable.

With all of my initial assumptions proven wrong, it was time to dive into what exactly was occurring when Spreedly initialized the managed input fields.

window.Spreedly

Before we start looking through, examining, and debugging code from this endeavor, let's take a moment to talk about how the Spreedly JavaScript library works at a high level.

The library must be included via a <script></script> tag in your html and is not available to be installed as a dependency via npm or yarn. By including this script, the Spreedly object will become available to use and access on the window.

<head>
  <!-- Other tags excluded for brevity -->
  <script src="https://core.spreedly.com/iframe/iframe-v1.min.js"></script>
</head>

Next, we must set up a series of event listeners. Spreedly's library emits events we can listen for and react to such as ready, errors, validate, etc. This gives us a way to tie in our own application logic to these events.

window.Spreedly.on('errors', console.error);

window.Spreedly.on('validate', validateForm);

window.Spreedly.on('ready', () => {
  // Because the elements Spreedly targets in our application/form are just unstyled divs,
  // we must format them through their UI API.
  window.Spreedly.setFieldType('number', 'text');
  window.Spreedly.setNumberFormat('prettyFormat');
  window.Spreedly.setPlaceholder('number', '···· ···· ···· 1234');
  window.Spreedly.setPlaceholder('cvv', '000');

  // spreedlyFormStyles is an array of our custom styles to apply to the Spreedly-managed
  // fields that must be applied via `window.Spreedly.setStyle(field, style)`
  spreedlyFormStyles.forEach(([field, style]) =>
    window.Spreedly.setStyle(field, style)
  );
});

Finally, we can initialize the library by calling window.Spreedly.init from the Lifecycle API. We pass init our Spreedly environment key and an object with key values that include the element IDs of the unstyled divs, which the library should manage and transform into form inputs for us. Another important thing to note is that the init function will emit the ready event once it has bootstrapped our controlled form elements.

window.Spreedly.init('super-secret-spreedly-environment-key', {
  numberEl: 'spreedly-cc-number',
  cvvEl: 'spreedly-cvv',
});

From this point on we should be good to go and able to take full advantage of the remaining features and APIs provided by Spreedly such as tokenization and recaching.

The Missing console.log

Now that we have a base understanding of how the Spreedly library works, the available APIs, and how we can interact with it, let's jump into debugging this. What do we know so far that might give us a good starting point? We know that when we call init on the Spreedly client it uses the string element IDs of our credit card number and CVV divs to somehow start controlling and managing them for us. We also know that once the initialization process is complete that it will emit the ready event. With these two things in mind, my initial instinct was that something was failing during the call to init or that we were failing to properly register our event handlers.

My strategy for testing this hypothesis would rely on one of the most timeless debugging tools in our toolbox; console.log. I would be placing a series of log statements in our application and then examining the order and output of both when our application was running normally and also when it was running in Cypress. I chose to add log statements in the following places: one when we register our event handlers, one when we call init, and one when we receive the ready event.

console.log('adding event handlers');

window.Spreedly.on('errors', console.error);

window.Spreedly.on('validate', validateForm);

window.Spreedly.on('ready', () => {
  // Spreedly styling API code excluded for brevity
  console.log('ready');
});

console.log('initializing');

window.Spreedly.init('super-secret-spreedly-environment-key', {
  numberEl: 'spreedly-cc-number',
  cvvEl: 'spreedly-cvv',
});

Let's examine the output in the console from both contexts.

Running Normally

adding event handlers
initializing
ready
>

Cypress

adding event handlers
initializing
>

This confirmed my suspicion that something was going awry during the initialization process and I decided to double check the Spreedly docs for anything that may be helpful. I ended up finding the reload function as part of the Lifecycle API. Calling this would reinitialize the form and would also emit the ready event. I wanted to call this from the console in Developer Tools after the application had loaded and see what happened.

Because our application runs in an iframe, I didn't initially have access to the Spreedly client so I quickly added this line of code for testing purposes:

window.parent.Spreedly = window.Spreedly;

Let's see what happens when we call reload.

Running Normally

> window.Spreedly.reload()
adding event handlers
ready

Cypress

> window.Spreedly.reload()
adding event handlers

At this point I was feeling confident that the bug was not in our own application code. This would require taking a closer look at what Spreedly is doing under the cover of its init function and trying to resolve why it wasn't firing the ready event.

Down The Rabbit Hole

To continue troubleshooting this bug, it would require jumping into Spreedly's third party script(s), which are being served to us minified. To do this, we need to open our browser's Developer Tools and head over to the Sources tab. From there, we just need to look under the Page pane. Here we'll find a tree showing us a hierarchy of windows and the sources that they load. You can see the various iframes in this view as denoted in the tree by the window icon and the names top, localhost/, spreedly-cvv-frame-7206 (cvv-frame.html), and spreedly-number-frame-7206 (number-frame.html). These are a one-to-one mapping of the iframe boundaries we looked at previously.

Let's open iframe-v1.min.js and see what we've got.

At first glance this looks pretty intimidating and like it would be a nightmare to attempt to read through and parse. Fortunately for us, Developer Tools gives us a way to "Pretty Print" and format this minified code by clicking the {} in the bottom left-hand corner.

Now that we have some formatted code to work with we can start searching for the init function in the file. After jumping through a few instances of search results we hit the following block of code with the function definition:

(this.init = function (t, e) {
  this.isLoaded() && this.unload(),
    e && e.source && (this.source = e.source),
    e && e.numberEl
      ? (this.numberTarget = e.numberEl)
      : (this.numberTarget = m('data-number-id')),
    e && e.cvvEl
      ? (this.cvvTarget = e.cvvEl)
      : (this.cvvTarget = m('data-cvv-id')),
    (this.environmentKey = t || m('data-environment-key')),
    (this.numberFrameId = 'spreedly-number-frame-' + this.uniqueId),
    (this.cvvFrameId = 'spreedly-cvv-frame-' + this.uniqueId),
    this.addIframeElements();
}),

The init function is taking our input (the environment key and object containing the element ids for both fields), doing some validation, adding those values to its own state, and then calling this.addIframeElements.

(this.addIframeElements = function () {
  var t = a(p);
  (t.id = this.numberFrameId),
    (t.name = this.numberFrameId),
    t.setAttribute('src', b(g, w(this.source))),
    document.getElementById(this.numberTarget).appendChild(t);
  var e = a(d);
  (e.id = this.cvvFrameId),
    (e.name = this.cvvFrameId),
    e.setAttribute('src', b(v, w(this.source))),
    document.getElementById(this.cvvTarget).appendChild(e);
}),

This function is creating the <iframe> elements for the child iframes (where each managed/controlled input lives), setting the src attribute, and finally appending it to the document. This causes the subsequent Spreedly sources (as we saw in the Pages pane above) to be fetched and loaded. Interestingly, there was no code that emitted the ready event here. Before we dive into the sources that are loaded by the child frames, let's do a quick search of iframe-v1.min.js for the string 'ready' to see if we can find any clues.

We get a hit!

(this.buildMessageHandler = function () {
  var t = this;
  return function (e) {
    if (
      ('string' == typeof e.data || e.data instanceof String) &&
      e.origin === h &&
      t.checkUniqueId(e.data.substring(0, 4)) &&
      t.isLoaded()
    ) {
      var n, r = e.data.substring(4);
      if ('frameLoaded:' === r);
      else if ('iframesReady' === r)
        t.source && t.sendMessage('source: ' + t.source),
          t.emit('ready');
// ... excluded for brevity

Here we find the buildMessageHandler function. This is a function that creates and returns a separate message handler function that listens for and parses a series of internal events that get passed on the data key of an event object. Also in this file, buildMessHandler is called and used to attach an event handler for handling message events.

(this.messageHandler = this.buildMessageHandler()),
i.addListener(window, 'message', this.messageHandler),

The message event is a mechanism that can allow for communication and message passing between different browser contexts (i.e. windows, iframes, tabs, etc). This is how Spreedly is communicating data and events between their various iframes.

Jumping back to the buildMessageEvent function, we find that when the iframesReady event is handled, the Spreedly client, in turn, emits its own ready event. This is the event that is actually never emitting for us. We can probably deduce that this event fires when the two child iframes are initialized and ready, and that the bug we're hunting must reside in the code responsible for emitting it. Performing another search for the string 'iframesReady' we come up empty handed. It must be getting emitted from one of the child iframes. Let's jump into number-frame-1.56.min.js first and do a search there.

Sure enough, we find a result. The event is being emitted from a function called setCvvWindow.

setCvvWindow: function (e) {
  var t = e || this.getCvvWindow();
  (t.onerror = this.consoleError),
    (this.cvvField = t.document.getElementById('cvv')),
    (this.cvvLabel = t.document.getElementById('cvv_label')),
    (this.cvvForm = t.document.getElementById('cvv-form')),
    (this.cvvInputListener = this.buildInputListener()),
    n.addInputListener(this.cvvField, this.cvvInputListener),
    (this.cvvForm = t.document.getElementById('cvv-form')),
    this.message('iframesReady');
},

Reading through this function it would seem that the credit card number iframe gets a reference to the CVV window, adds an input listener, and then emits the iframesReady event. Let's see if we can find where setCvvWindow is being called from.

The only other reference to this function in number-frame-1.56.min.js is the following code that attaches it to the window as setUpCvv which then is not called or found anywhere else in the file.

(window.setUpCvv = function () {
  n.setCvvWindow();
}),

Doing a quick search for setUpCvv in iframe-v1.min.js also yields no results. We must go deeper. Looking at the sources loaded in the cvv iframe, we find that only an html file named cvv-frame.html is fetched and loaded. Let's open it up and see if we find any references to setUpCvv.

Voilà! We find it under an inline function named establishCommunication.

var establishCommunication = function () {
  try {
    if (
      window.parent.frames[numberWindowName] &&
      window.parent.frames[numberWindowName].setUpCvv
    ) {
      window.parent.frames[numberWindowName].setUpCvv();
      clearInterval(messageInterval);
    }
  } catch (err) {}
};

So what's going on here? When establishCommunication is invoked, it checks the parent window's array of iframes for the sibling credit card number window/iframe (window.frames[numberWindowName]), and then also checks for the existence of the setUpCvv function on that window.

If those conditions are met, the CVV window/iframe calls the sibling window's setUpCvv function and then clears an interval named messageInterval if no errors occur.

Where is this interval being created and started? Doing a quick search we discover the following line of code which creates an interval that calls establishCommunication every five milliseconds.

messageInterval = setInterval(establishCommunication, 5);

Knowing what we know now, we can narrow down where the bug is occurring. Either the if condition in establishCommunication is never true, meaning that there is an issue loading the credit card number iframe, or an error is happening in the the setUpCvv function (and subsequently setCvvWindow) before the iframesReady event is being emitted.

We can easily test this by placing a debugger in the script.

The One-Line Difference

A wise place to put a debugger in this instance might be on the first line of the setCvvWindow function in number-frame-1.56.min.js.

var t = e || this.getCvvWindow();

If the condition in cvv-frame.html's establishCommunication is evaluating to true, we know that this function should be called and we would certainly hit this point in the code. This could help us rule out one of the potential sources of the bug. In addition, we would expect that the debugger would instantly hit/catch when put on that line, since we know that iframesReady is never being emitted and therefore the messageInterval in establishCommunication would never get cleared (meaning it's constantly running every 5ms).

Sure enough, the debugger hits as soon as we place it on that line. We need to take a step to the next line and evaluate the value of t.

Aha! The value of t is undefined! We know that both setUpCvv and setCvvWindow are not invoked with an argument, so that rules out a problem with the variable e. Because t is undefined the next line where we try to set t.onerror is throwing a type error: Uncaught TypeError: Cannot set property 'onerror' of undefined.

The problem must lie within this.getCvvWindow,which is returning undefined.

getCvvWindow: function () {
  return window.self.frames[this.cvvWindowName];
},

Strange. Sure enough, adding a debugger to this line and checking window.self.frames in the console returns undefined. Why would this line of code be causing issues in Cypress (yet not Storybook, which if you recall, runs in similar conditions as Cypress) but not when our application is running normally?

The next logical step was to put a debugger on the same line and hit it while our app is running normally outside of Cypress.

This is where things took a turn and got truly bizarre.

When I opened number-frame-1.56.min.js and jumped to getCvvWindow this is what I found.

getCvvWindow: function () {
  return window.parent.frames[this.cvvWindowName];
},

If it isn't immediately clear (it wasn't for me either), the code is different!

Running Normally

getCvvWindow: function () {
  return window.parent.frames[this.cvvWindowName];
},

Cypress

getCvvWindow: function () {
  return window.self.frames[this.cvvWindowName];
},

window.parent vs window.self.

What?! How?! 🤯

Was Spreedly somehow conditionally sending different source code? That hardly made any sense, but if so, why was this the only line in the entire file that was different and what was the condition that would cause it? Was it for some obscure security purpose? I was puzzled to say the least.

A Closed GitHub Issue

Still in disbelief, I ended up bringing in two of my coworkers, Brandon Konkle and Cesar Perez, to take a look at what I had discovered. Their minds were blown too. None of us could figure out why or how this was happening.

We decided the next logical step to take would be to compare the network requests being made for Spreedly's script(s) to see if there was any sort of difference that could be causing different source code to be returned in response. We did this by navigating to the Network tab in Developer Tools, right clicking on the request for iframe-v1.min.js, going to Copy, and then selecting Copy as cURL. We then pasted both cURL requests in a text editor to compare.

Scanning for differences in the two requests, only one thing stood out to us—we were sending a cookie in the one that was working. Could Spreedly be using this cookie value to send a script with a one-line difference on the fly? This seemed farfetched, but it was the only working theory we had.

We updated our Cypress configuration and tests to preserve cookies between runs and manually added the cookie to our request with an identical value.

No dice. The test was still failing and the script was still incorrect and being served with window.self rather than window.parent.

As a confidence check, we decided to run the cURL request we had copied from the browser running Cypress to make sure it was returning the broken copy of the code to our terminal.

Another plot twist.

To our surprise, it returned the correct code. This compromised our working theory that Spreedly was somehow conditionally sending us different source code.

We reset our gaze upon Cypress. We now knew that we were only getting the altered source code in the browser (as confirmed by cURL) and while Cypress was running our application. Was it possible that Cypress was rewriting a third-party's code at runtime? This seemed like the most out-there theory we had yet, but what else could it be?

Off to Google we went.

We quickly found issue #2664: A closed issue on Cypress' GitHub repo where other users were also running into similar issues. We also found this comment from a user named aldocd4. They claimed that adding the following configuration to our cypress.json resolved the issue for them.

{
  "modifyObstructiveCode": false
}

We gave it a shot, crossed our fingers, and reran our tests.

Success! Sure enough, it had worked! 🎉

Conclusion

In the end, Cypress had been the culprit all along. The modifyObstructiveCode configuration option is one that none of three of us had ever heard of before or encountered, and it turns out that it is enabled by default. Here are some snippets from the documentation on it.

With this option enabled - Cypress will search through the response streams coming from your server on .html and .js files and replace code that matches the following patterns.

These techniques prevent Cypress from working, and they can be safely removed without altering any of your application’s behavior.

Additionally it’s possible that the patterns we search for may accidentally rewrite valid JS code. If that’s the case, please disable this option.

Well, as it turns out, Cypress was rewriting valid code (and third-party code at that) and breaking our application's behavior. On top of that, had it not been for our efforts of thoroughly digging through the Spreedly source and exhausting all of our other means of debugging, we would have never figured out what was going on as Cypress gave no indication that it was performing this behavior at all. Fortunately, now, modifyObstructiveCode is disabled and our tests are happily passing and working again.

Hopefully you've enjoyed my story of tracking down and resolving this bug and maybe learned something along the way. It was, without a doubt, one of the most interesting, challenging, and dare I say it, fun, bugs I've worked on in my career and was incredibly rewarding to solve in the end.

If you've ever run into a bug like this, reach out on Twitter. I'd love to hear about it.

Thanks for reading and happy debugging!