Should you Delete or Soft Delete?

Sometimes a user requests that all the data be deleted permanently, you can't just make another entry elsewhere that they requested to be deleted and capture data on that event because you will be violating their right to be forgotten. Sometime a delete is just a delete.

about 15 minutes of real, actual work.

I'm pretty sure if I'm Peter I want to hard delete all TPS reports.

Thanks for the video! Choosing between event sourcing and crud seems to be a difficult job. How should I know when to use which? And later down the line, when I realize that I should’ve gone with the other method, how should I make the change? Changing from event sourcing to crud seems pretty simple, as I already have all the data. But how should I change from crud to event sourcing? Or should I just event source everything to be safe?

How do you handle mis-keyed data in a task based system? Let's say an employee moved, and you go through the workflow of entering a new address. You click save, the system kicks off an employee moved event and everything is great. Then you noticed you miskeyed their street name. Do you have another employee moved event? They didn't really move twice. Do you have an AddressCorrected event? At that point do you have to support task based AND crud based events for every piece of data?

Nah... you should cover GDPR where users has rights to wipe out their data. So you should wipe out all sensitive marked data. You can't keep it in original form.

So you would add data with more information (i.e. EmployeeTermination and EmployeeHiring) instead of toggling an isDeleted flag because the facts of what happened might be important.

so what you are saying hireEvent = add to db, and terminateEvent = update db... so your answer is softdelete.

Soft deletes are the bane of my existence. Got a system I’ve inherited where the soft delete flag means something different depending on the entity it’s on. And query/report performance is crap because it’s got to be included in every one

Soft deleting user info is illegal in Europe

Derek do you use C4 diagrams? If so would you ever do a primer on them?

yean totally agree that events are more informative than crud, but I feel like the storage of events could be challenging. do you have any advice on that? let's say one entity is a row in the sql db, if its crud, no matter how many events happened in the domain, cuuud ends up with d, one row less for the table, however, it would be five events to stored if we do event sourcing.

I can think of several occasions where you should actually hard delete things. The first one that comes to mind is: If I click "Terminate My Account" on facebook for instance, Facebook should hard delete all of the media that I uploaded there.

Now the question is: Will facebook actually delete it? I don't know.
But should facebook hard delete it? Yes.

What about the right to be forgotten? instances when you ask the company to delete your data

A lot of your videos hit on a very important topic without actually explicitly getting into it. It's very important to have a competent product owner who can focus on delivering their team with meaningful acceptance criteria for actual business cases instead of things that look like CRUD operations. This is often a challenge with businesses that do a poor job aligning teams with business functionality and not technical considerations.

Peter Gibbons: what would you say you do here? do you deal with the gd customers so the engineers don't have to? do you have people skills?

I totally agree with you. There are even some regulations and laws that forbid data deletion and enforce keeping data history.

What about GDPR?

A great example of domain specific language usage!

Even when from a business standpoint you don't think of hard deletes, regulations like the GDPR and right to be forgotten laws would still requires systems to be able to hard delete. How do you handle such regulatory requirements?

Do you delete user related data from even store to comply with GDPR?

​ @CodeOpinion I've read the comments :D Please stop explaining purpose of this video to everyone. It's like riding a bicycle with inverted handlebars - your brain needs to adopt first.

One notable exception: if your business wants to comply with European privacy laws and receives a request to delete PII for an individual, then deletion truly is in line with a business process.

Doesn't this make your life as a developer much more difficult? For example we use soft delete, and in our EF database context we filter out based on IsDeleted == false as the default. Only when we explicitly requested deleted data is it returned from the API. This means we don't have to think about deleted data. If we went with your approach, we would have to have seperate logic for filtering out deleted records for every entity.

I can 100% support this mind set. I’ve worked in large scale and super small projects and in my experience I’ve started evolving away from a soft delete an instead use some kind of enumeration or status code id usually stored in a database of some sort to define that current status of data. I think a lot of the frustration caused here is generally because these type of thinking does require user interfaces and queries to be a bit smarter than we might like but I’ve found it’s usually worth it.

It would be helpful if you describe what the database schema and database changes would be for the evented version. It’s easy to see how soft delete works because you showed the schema with the IsDeleted column and how/when it is updated. But it’s not clear on what the db schema is and how data is managed in the evented version.

What if you Aggregate Root has a collection of items, and then somewhere in your code you write AggregateRoot.Collection.Remove(item)? In my case the collection is lets say an Assessment and the collection is Reviewers. But I want to remove one of the Reviewers of the Assessment. Lets say I am using EF and SQL database, What would be the best way to handle that?

Answering the question of whether to hard delete vs soft delete with the the notion of "adding" things doesn't make any sense. It still doesn't answer the question on whether you should use one or the other because the it's irrelevant to the question, with no alternative provided, and assumes extremely contextual business application requirements that do not apply universally! There's lots of cases where recording a chronological order of events is not the right solution. Not a great answer to the question IMHO.

"Why store all these events and data taking up space?" - Because of auditing. "But I don't need that" - So be it, this is just a concern that might become important to your business later down the road. You need to know that this is a way of thinking about your system. And you don't have to go all in on event sourcing for this either.

And then GDPR became a thing for many of us, so we cannot just keep data around indefinitely. Eventually you will have to hard delete if it applies to your business.

Expanding on the subject of soft-delete, do you believe that adding "created by", "created at", "updated by", "updated at", "deleted by" and "deleted at" columns to your database is redundant when you can grab that information from the event store?

Very easy situation to get into. The most annoying repercussion I've come across is having to omit IsDeleted entities throughout the entire application, including navigation properties. If you're selecting from the db at the root entity level, it's very difficult to deal with in a large app.

Soft delete is useful for adding a versioning mechanism. Also an alternative is to use temporal tables.

Looks like this is a good approach for nosql? how do you implement something like this for SQL?