undefined | Better HN

0 pointsaleksiy1231y ago0 comments

We seem to be using different definitions of stub and fake. What you call a stub is generally known as a Fake.

https://stackoverflow.com/questions/3459287/whats-the-differ...

Stubs do not react based on their input parameters. However, even fakes have the issue of fault injection.

I don't want to just test valid inputs. I want to make sure I pass the correct inputs for the request parameters through to the dependencies. If pass 100.00 in the request but in my impl I pass 0.00 to the stub. The test will pass because the input is valid. But the implementation is not correct.

Also, mockery is just an alternative mocking api. There is nothing special here.

How would you test the case where the quota update fails with a stub? Here is mockery example again generated with chatgpt.

    func TestHandleTransaction_QuotaUpdateFails(t *testing.T) {
      // Create the mock services
      mockAuth := new(mocks.AuthService)
      mockPayment := new(mocks.PaymentService)
      mockStorage := new(mocks.StorageService)
      mockQuota := new(mocks.QuotaService)
      mockNotification := new(mocks.NotificationService)

      // Setup expectations
      mockAuth.On("Authenticate", mock.Anything).Return("12345", nil)
      mockQuota.On("CheckQuota", "12345").Return(true, nil)  // Quota check passes
      mockQuota.On("UpdateQuota", "12345", mock.AnythingOfType("int64")).Return(errors.New("quota update failed"))  // Quota update fails
      // No need to mock payment and storage as they should not be called if quota update fails

      // Create the server with mock services
      server := &Server{
          Auth:            mockAuth,
          Payment:         mockPayment,
          Storage:         mockStorage,
          Quota:           mockQuota,
          Notification:    mockNotification,
      }

      // Create a test request
      transaction := TransactionRequest{
          UserID:    "12345",
          AuthToken: "valid-token",
          ProductID: "product123",
          Price:     100.0,
          Currency:  "USD",
          File:      []byte("file data"),
      }
      requestBody, _ := json.Marshal(transaction)
      request := httptest.NewRequest(http.MethodPost, "/transaction", bytes.NewReader(requestBody))
      responseRecorder := httptest.NewRecorder()

      // Call the endpoint
      server.HandleTransaction(responseRecorder, request)

      // Check the results
      assert.Equal(t, http.StatusInternalServerError, responseRecorder.Code)
      response := TransactionResponse{}
      json.NewDecoder(responseRecorder.Body).Decode(&response)
      assert.Equal(t, "error", response.Status)
      assert.Contains(t, response.Message, "quota update failed")

      // Assert that all expectations were met
      mockAuth.AssertExpectations(t)
      mockQuota.AssertExpectations(t)
      // Assert no calls to payment and storage
      mockPayment.AssertNotCalled(t, "Charge", mock.AnythingOfType("float64"), mock.AnythingOfType("string"), mock.AnythingOfType("string"))
      mockStorage.AssertNotCalled(t, "Upload", mock.AnythingOfType("[]uint8"), "bucket-name", "key-name")
      // No notification should be sent
      mockNotification.AssertNotCalled(t, "SendNotification", "12345", mock.AnythingOfType("string"))
    }

0 comments

randomdata1y ago

> What you call a stub is generally known as a Fake.

I willingly accept your unconventional usage if it greases discussion, but the dictionary is right there. It records how most people use words. What mockery gives appears to be what most people consider a fake. It clearly does not fit the definition of mock. It arguably matches the definition of stub and I think you could reasonably call it that, but I posit that doesn't tell the whole story, which is no doubt why fake emerged.

That said, I don't know what your definitions are. You seem to flail around with them.

> Stubs do not react based on their input parameters.

All of mocks, stubs, and fakes react based on input...

> How would you test the case where the quota update fails with a stub?

... Although this seems like a poor example. It is not clear if the failure state is due to invalid input or due to some other issue.

This, I must say, makes for a really bad test. Tests are, first and foremost, the contract and documentation for your users. They are written so that other people can learn about the system and know what they can depend on during use. The self-validation offered by testing is there merely to ensure that what is written in the contract is true. –– Not that we were expecting anything more from ChatGPT, but this does, interestingly, indicate yet another problem with mockery. I hadn't even picked up on that one earlier. Glad we were able to keep the discussion going to learn more!

But, importantly, we lack necessary information to respond to your question. Perhaps you can clarify the intent here?

aleksiy123OP1y ago

You asked for code and I gave you a fairly real world example of where mocking is useful.

My suggestion is for you to provide code how you would test the same handler code and scenario using your methodology.

Show me a test that checks the error handling of quota check. Pretend the quota service is experiencing an outage on the second request you make to it.

randomdata1y ago

The example was okay, but incomplete. It seems we have enough information now, so:

    type quotaNetworkFailure struct{ quotaSuccess }
    func (quotaNetworkFailure) UpdateQuota() (bool, error) { return false, quota.ErrNetworkFailure }

    func TestTransaction_QuotaUpdateNetworkFailure(t *testing.T) {
        tx := &Transaction{
            Auth:         authSuccess{},
            Payment:      paymentSuccess{},
            Storage:      storageSuccess{},
            Quota:        quotaNetworkFailure{},
            Notification: notificationSuccess{},
        }
        _, err := tx.Request("12345", "valid-token", "product123", 100.0, "USD", []byte("file data"))
        if !errors.Is(quota.ErrNetworkFailure, err) {
            t.Errorf("unexpected error: %v", err)
        }
    }

While still contrived, surely you agree that is much more understandable, not to mention a whole lot easier to write? Now we know why something might fail, and the reader learns what they should look for when handing a network failure.

Yes, okay, you got me. That isn't the HTTP service anymore. But it should have never been. The transaction processing is not an HTTP concern. In fixing that, now you can make the HTTP end of things far less obtuse:

    var errGeneralError = errors.New("general error")
    type transactionFailure struct{}
    func (transactionFailure) Request(id, token, productID string, amount float64 /* yikes */, currency string, file []byte) {
        return false, errGeneralError
    }

    func TestBuy_Failure(t *testing.T) {
        srv := httptest.NewServer(&Server{
            Transaction: transactionFailure{},
        })    
        defer srv.Close()

        c := client.New(srv.URL)
        _, err := c.Buy("product123")
        if !errors.Is(transaction.ErrFailed, err) {
            t.Errorf("unexpected error: %v", err)
        }
    }

But, this is still not a good example of input. Input doesn't matter when the network fails. Let's say we want to cover the case where the product ID being purchased doesn't exist instead:

    type transactionNoProduct struct{}
    func (transactionNoProduct) Request(id, token, productID string, amount float64 /* yikes */, currency string, file []byte) (bool, error) {
        if productID != "product123" {
            return true, nil
        }
        return false, transaction.ErrProductNotFound
    }

    func TestBuy_ProductNotFound(t *testing.T) {
        srv := httptest.NewServer(&Server{
            Transaction: transactionNoProduct{},
        })    
        defer srv.Close()

        c := client.New(srv.URL)
        _, err := c.Buy("product123")
        if !errors.Is(transaction.ErrProductNotFound, err) {
            t.Errorf("unexpected error: %v", err)
        }
    }

There is still plenty of room for improvement here that I will cut short as who cares for an HN comment, but man, I don't see how anyone can think that mockery monstrosity is preferable to... anything else. Where is it actually useful?

1 more reply

j / k navigate · click thread line to collapse

0 pointsaleksiy1231y ago0 comments

We seem to be using different definitions of stub and fake. What you call a stub is generally known as a Fake.

https://stackoverflow.com/questions/3459287/whats-the-differ...

Stubs do not react based on their input parameters. However, even fakes have the issue of fault injection.

Also, mockery is just an alternative mocking api. There is nothing special here.

How would you test the case where the quota update fails with a stub? Here is mockery example again generated with chatgpt.

    func TestHandleTransaction_QuotaUpdateFails(t *testing.T) {
      // Create the mock services
      mockAuth := new(mocks.AuthService)
      mockPayment := new(mocks.PaymentService)
      mockStorage := new(mocks.StorageService)
      mockQuota := new(mocks.QuotaService)
      mockNotification := new(mocks.NotificationService)

      // Setup expectations
      mockAuth.On("Authenticate", mock.Anything).Return("12345", nil)
      mockQuota.On("CheckQuota", "12345").Return(true, nil)  // Quota check passes
      mockQuota.On("UpdateQuota", "12345", mock.AnythingOfType("int64")).Return(errors.New("quota update failed"))  // Quota update fails
      // No need to mock payment and storage as they should not be called if quota update fails

      // Create the server with mock services
      server := &Server{
          Auth:            mockAuth,
          Payment:         mockPayment,
          Storage:         mockStorage,
          Quota:           mockQuota,
          Notification:    mockNotification,
      }

      // Create a test request
      transaction := TransactionRequest{
          UserID:    "12345",
          AuthToken: "valid-token",
          ProductID: "product123",
          Price:     100.0,
          Currency:  "USD",
          File:      []byte("file data"),
      }
      requestBody, _ := json.Marshal(transaction)
      request := httptest.NewRequest(http.MethodPost, "/transaction", bytes.NewReader(requestBody))
      responseRecorder := httptest.NewRecorder()

      // Call the endpoint
      server.HandleTransaction(responseRecorder, request)

      // Check the results
      assert.Equal(t, http.StatusInternalServerError, responseRecorder.Code)
      response := TransactionResponse{}
      json.NewDecoder(responseRecorder.Body).Decode(&response)
      assert.Equal(t, "error", response.Status)
      assert.Contains(t, response.Message, "quota update failed")

      // Assert that all expectations were met
      mockAuth.AssertExpectations(t)
      mockQuota.AssertExpectations(t)
      // Assert no calls to payment and storage
      mockPayment.AssertNotCalled(t, "Charge", mock.AnythingOfType("float64"), mock.AnythingOfType("string"), mock.AnythingOfType("string"))
      mockStorage.AssertNotCalled(t, "Upload", mock.AnythingOfType("[]uint8"), "bucket-name", "key-name")
      // No notification should be sent
      mockNotification.AssertNotCalled(t, "SendNotification", "12345", mock.AnythingOfType("string"))
    }

0 comments

randomdata1y ago

> What you call a stub is generally known as a Fake.

That said, I don't know what your definitions are. You seem to flail around with them.

> Stubs do not react based on their input parameters.

All of mocks, stubs, and fakes react based on input...

> How would you test the case where the quota update fails with a stub?

... Although this seems like a poor example. It is not clear if the failure state is due to invalid input or due to some other issue.

But, importantly, we lack necessary information to respond to your question. Perhaps you can clarify the intent here?

aleksiy123OP1y ago

You asked for code and I gave you a fairly real world example of where mocking is useful.

My suggestion is for you to provide code how you would test the same handler code and scenario using your methodology.

Show me a test that checks the error handling of quota check. Pretend the quota service is experiencing an outage on the second request you make to it.

randomdata1y ago

The example was okay, but incomplete. It seems we have enough information now, so:

    type quotaNetworkFailure struct{ quotaSuccess }
    func (quotaNetworkFailure) UpdateQuota() (bool, error) { return false, quota.ErrNetworkFailure }

    func TestTransaction_QuotaUpdateNetworkFailure(t *testing.T) {
        tx := &Transaction{
            Auth:         authSuccess{},
            Payment:      paymentSuccess{},
            Storage:      storageSuccess{},
            Quota:        quotaNetworkFailure{},
            Notification: notificationSuccess{},
        }
        _, err := tx.Request("12345", "valid-token", "product123", 100.0, "USD", []byte("file data"))
        if !errors.Is(quota.ErrNetworkFailure, err) {
            t.Errorf("unexpected error: %v", err)
        }
    }

    var errGeneralError = errors.New("general error")
    type transactionFailure struct{}
    func (transactionFailure) Request(id, token, productID string, amount float64 /* yikes */, currency string, file []byte) {
        return false, errGeneralError
    }

    func TestBuy_Failure(t *testing.T) {
        srv := httptest.NewServer(&Server{
            Transaction: transactionFailure{},
        })    
        defer srv.Close()

        c := client.New(srv.URL)
        _, err := c.Buy("product123")
        if !errors.Is(transaction.ErrFailed, err) {
            t.Errorf("unexpected error: %v", err)
        }
    }

But, this is still not a good example of input. Input doesn't matter when the network fails. Let's say we want to cover the case where the product ID being purchased doesn't exist instead:

    type transactionNoProduct struct{}
    func (transactionNoProduct) Request(id, token, productID string, amount float64 /* yikes */, currency string, file []byte) (bool, error) {
        if productID != "product123" {
            return true, nil
        }
        return false, transaction.ErrProductNotFound
    }

    func TestBuy_ProductNotFound(t *testing.T) {
        srv := httptest.NewServer(&Server{
            Transaction: transactionNoProduct{},
        })    
        defer srv.Close()

        c := client.New(srv.URL)
        _, err := c.Buy("product123")
        if !errors.Is(transaction.ErrProductNotFound, err) {
            t.Errorf("unexpected error: %v", err)
        }
    }

1 more reply

j / k navigate · click thread line to collapse