Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Model Attributes #98

Closed
peelman opened this issue Nov 30, 2013 · 62 comments
Closed

Custom Model Attributes #98

peelman opened this issue Nov 30, 2013 · 62 comments
Assignees
Milestone

Comments

@peelman
Copy link

peelman commented Nov 30, 2013

@snipe has mentioned this in another ticket, but I was hoping to get some discussion going on it and define the idea and storage model some.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@snipe
Copy link
Owner

snipe commented Nov 30, 2013

Every model (I think this would be at the model level, makes the most sense for my use-cases, but open to discussion - could maybe work on categories instead?) could have custom fields which would then be associated with all assets within that model. Almost like a form-builder. So the ASUS 23" Monitor model might not have any custom fields, but the MacBook Pro 13" Retina Late 2012 might have custom fields like:

  • CPU
  • RAM
  • OS
  • etc

Every time a new asset is created that belongs to that model, the add asset form would dynamically update (or there would be a page 2 for asset creation) that would pull those fields.

Fields can be marked as required per model, so an asset can't be created until those values are filled in.

The obvious, inelegant solution would be to dynamically add column names. This is how Tracmor does it, and it works well, despite making a little part of my soul die every time. The advantage to this method is that searching on these fields is pretty straightforward, whereas with more complicated join tables, searching can get to be a bit of a bitch.

@snipe
Copy link
Owner

snipe commented Nov 30, 2013

Ack - hit submit too soon. Meant to add that I'm not sure which way I'd want to handle this, the normalized, join-heavy way or the de-normalized, inelegant way. Searching is definitely a concern - I might want to do a search to see all of the RTD pieces I have that have 16GB of RAM, for example. Obviously, those types of searches aren't impossible using properly normalized tables, but they definitely get more complicated.

@peelman
Copy link
Author

peelman commented Nov 30, 2013

From a growth perspective, the join method is the way to go, whatever it does to make searching difficult. Redmine's system has worked well for me in the past (database schema here), but I don't know what it would be like layering searching atop that.

Being able to add custom attributes at the model level works well enough for me, but if we can generalize that attribute system enough, there's no reason you couldn't make it available to individual assets/licenses/certificates as well. Allowing one-offs like that definitely opens up some headaches, but it also adds another layer of flexibility too. Not saying I am a particular advocate of that, but it should definitely be something to keep in mind when designing this, because re-applying custom attributes beyond hardware assets and licenses will be beneficial down the road I'm sure.

I'll add that when I am looking at this, I don't care about the machine specs as much as I care about the unique attributes:

  • MAC Addresses (I can't think of a system that doesn't have at least 2, several of our servers have 6 or more)
  • Service Tag (or whatever the non-Dell equivalent of that is)
  • AppleCare ID
  • Product Number
  • IMEI
  • Accessories (trackpad, keyboard, thunderbolt adapters, extra chargers, etc.)

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

Yes, as stated, there are advantages to both ways. The schema would be abstracted the same way the asset log is, so licenses, hardware, etc could use it. But searching is a very real (and I'd argue pretty common) use case, so it can't be ignored.

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

In my use cases, accessories wouldn't make sense for a use case here, as they would be covered by #16

@peelman
Copy link
Author

peelman commented Dec 1, 2013

We generally don't tag anything that doesn't have an (easily) visible serial number, so a lot of that would be happening for us.

At some point the question becomes when would we be better served by a true inheritance model (faux pax though gen-spec relationships are in the database world) and flesh out true classes for asset types, leaving a general purpose one for things that don't fit into the provided buckets. At this point I'm spitballing, but its an idea I've circled repeatedly since I first rebuilt SimpleAssets to suit my then-needs in 2008 (lots of working strategies in that project, but it was long dead even back then). A more abstract, expandable model is certainly a better goal on the face of it, but as you have pointed out, it makes searching (and many other facets) a hell of a lot harder to deal with down the road.

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

We don't typically tag things like keyboards, etc, but I'm considering starting, just so I have some way of knowing how many of x we have, who has x, etc. It's obviously less critical (Hmm... $30 keyboard versus $3k MBP...) but it's something of an annoyance for me. I never know if we have spare peripherals, etc.

One of those use cases is repair work as well. The purpose of #16 is partly to also be able to track total expense of a machine, repair record etc. One laptop has had 4 harddrive swap, etc. The hard-drive is currently in this laptop, but it may end up in another laptop at some point. I want to be able to track cost for those new component parts, and what's been swapped out when.

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

I should mention, we don't typically tag hard drives, even though they have a serial number.

@peelman
Copy link
Author

peelman commented Dec 1, 2013

All of that would fall to the notes/comments section for me (or at least it has in the past). It serves as a status log, work log, etc. for each asset. New hard drives, motherboards, etc., all would get documented inside the ticket. If a new drive was installed it's serial would get logged, as well as the outgoing part (and what happened to it: was it destroyed?, how?on what date? etc).

At one point I based a system entirely off of redmine (which I started building a true asset management plugin for) and it worked rather well; treating each classification as a project, and each asset as a ticket, using custom fields to define particular datasets. It definitely wasn't designed for it, and it showed, but when all you have is a hammer...

This is all theoretical at our current place, but I'm going off of how we handed it in my past positions, trying to roll the best of each clusterfuck we had and leave behind as many of the nasty parts as a I can.

To achieve the TCO you're going for in #16, I wonder if you don't need to do something that is more dedicated to expense tracking, starting with the base price of a machine and adding entries for every upgrade/repair. Something akin to:

Expense

  • ID
  • asset_id
  • description
  • price
  • labor_hours
  • labor_price
  • warranty?
  • user_added
  • date_added

I guess you could add the requisite fields to the notes table; but this just feels cleaner and easier to manage going forward...it doesn't negate the value of having the ability to define parent/child relationships ala #16, just creates more work :(

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

I'd prefer to use quantifiable data for repair parts, versus just making a note. (Bear in mind, this is all v2, not even v1, IMHO.) I don't typically need to track hours and I may or may not track cost on repair items, so I see it less as an expense tracking thing than component tracking. I know myself (and my techs) will consistently forget to use any kind of time tracking, so if it was added, it should be a setting you could disable in the settings page and would be added for the benefit of other people using this system, not for me. (And I'm totally open to adding things I don't personally need, as long as there are clear uses cases for other folks - but they may not be my first priority to add.)

What the custom fields are doesn't really matter, IMHO, but I feel strongly that they're needed - and they would be extensible enough that they could tie into #79 as well. For all of this (very useful) back and forth, we still haven't come up with any decisions on how to handle custom fields tho. :P

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

The base requirement in my head is that models can have custom fields. You can define what kind of custom field (checkbox, text, etc - very basic form builder type of functionality) and whether a value is required. Tracmor does this and it's been very helpful.

Then when you go to create a new asset of Model X, as soon as you pick Model X from the drop down list, the new fields are created in that form. As I mentioned, Tracmor does this by creating columns on the fly, but that makes me feel oogie.

Knowing that search is a priority, we could also build out some kind of search table that basically has some kind of key-value store. It's janky, but it could simplify queries quite a bit. Basically a search index table that lets us keep the data normalization we so desperately crave, without making those search queries turn into something truly unholy (and slow).

@peelman
Copy link
Author

peelman commented Dec 1, 2013

custom_attribute
  • id
  • model_id (FK)
  • type_id (FK)
  • name
  • isRequired?
  • constraints/validator (JSON?)

    or

  • length
  • regex
custom_attribute_type
  • id
  • name
custom_attribute_value
  • id
  • custom_attribute_id
  • asset_id
  • value

If you wanted to allow Radio buttons it could be done through simple JSON parsing via a "values" column as well. The separate type lookup table could be eliminated (probably), and there are varying ways to handle constraints and validation.

There will probably need to be a weight or order value associated with it, to determine where in the Asset form it shows up.

At this point I'm leaning towards the Asset form being two parts, and having all custom entry done on the second page; from a lazy developer's perspective, that seems easiest.

@peelman
Copy link
Author

peelman commented Dec 1, 2013

One thing that does occur to me, do we / how do we want to handle adding custom attributes to licenses?

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

Re: your last note - we certainly could add it for licenses, I'm just not sure that it's needed. More importantly, I'm going to want this when the services/vendor section of the system is built out, so it makes sense to think of it beyond just assets. Basically, everything in the system is some sort of property - sometimes they live in different tables (assets, licenses) because there are business rules around all of it, but all of the goodies should be designed to extend to them. (One thing I've been pondering, as this thing progresses, is whether to pull licenses back into assets, and make all of they details (warranty, etc) as custom fields instead.)

Re: the pref note - I don't thin weighting is necessary. The custom fields appear in the order they were created. We could certainly add custom ordering over time if it were requested, but have never had a need for it and until I hear a compelling use case from someone who does, I don't see a reason to add it.

Your proposed schema makes sense to me, and was similar to what I had in mind. (I wasn't going to give validation beyond required/not required, but with the built-in validators in Laravel, it seems silly not to, as long as we can make the UX work.

Speaking of UX, I think I want to keep the asset creation to one page. We could do a wizard, but its just as easy with query to dynamically populate new fields based on what's selected from the models menu. Better (faster) UX overall, IMHO.

@uberbrady
Copy link
Collaborator

Hi guys - I was pondering this a little myself. I'm going to stay away from the nitty-gritty of database schemas and whatnot and instead talk through a use-case I imagined:

So I imagined that things like RAM, num CPU's, CPU speed, etc - would all be 'custom attributes'. And - just as @snipe mentioned - I could easily see wanting to search across different models to find "any computer with greater than 8GB RAM" - and have that spit out MacBook Pro Retina 15, Retina 13, Macbook Pro 13 - and whatever else I have on-hand with those specs.

So I was thinking about how to attach those extra attributes to the model and it struck me as repetitive and mistake-prone to have you have to go through and add each attribute to each model each time. Neither should you put it at the Manufacturer level - as Apple makes all kinds of things other than Macs and it might not be appropriate.

I was thinking you would have something like "asset types" or "custom attribute sets" or something like that - and when you're editing/creating your Model (e.g. Macbook Pro Retina 13), you can pick your "custom attribute set." (Or maybe just Asset Type, that could be simpler).

That way I don't have to manually add: RAM, integer, required, CPU, float, required, HD, float, required... et al to each model I end up specifying.

I can instead just pick from a drop-down (or something like that) and say "Asset Type: Computer" (or "Custom Attribute Set: Computer").

And so I guess I lied about database schema - I would modify custom_attribute_value instead to be like:

asset_id
attribute_id
value

with a unique index over (asset_id,attribute_id) (unless you guys can imagine a use case where you repeat an field a few times in an asset? BLECH) - and perhaps a covering index (might as well make it unique) over (asset_id,attribute_id, value) <- you can query from that and get a list of assets without ever even touching the custom_attribute_value table. I would lose the 'id' column since I don't think it adds anything. But @snipe will just shoot that idea down anyways so I don't know why I even propose it :) (she likes id's)

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

In my head, it would work tied to the category. So, category is laptops - every laptop has X, Y and Z attributes. Some laptops may have A and B attributes too. When you create the MODEL within the laptop category, the laptop attributes are provided for you, and you can then choose to include them with that model or not via checkboxes next to the custom fields you configured in the category. The default will be to include them, and we could hide all of that under an expandy "advanced options" section for simplified UI, but the category serves no real purpose otherwise. We can think of categories a "custom attribute collections" - but we won't call them that in the UI because no one will know WTF that means.

So models inherit custom fields from their category, and assets inherit custom fields from their models. This leaves it up to the user to organize their categories in a way that works for them, and prevents me from imposing my business rules on someone else's use cases.

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

To further illustrate (and I am unfortunately too lazy to mock something up in HTML):

CATEGORIES:

  • computers
  • mobile phones
  • tablets
  • monitors

In CATEGORY: computers, I might create the following custom fields:

  • RAM
  • RAM Slots
  • CPU
  • Num CPUs
  • Disk Size
  • (many more, if that's my jam, but you get the idea)

Then when I create a new MODEL - let's say a MBP 13" Retina, Late 2012, in that interface for creating that model, I have the same fields that currently exist (asset tag, asset name, etc) and then a collapsed section called "Advanced Options" or "Custom Fields". All custom fields for that category will be enabled by default, but you can easily disable them if you'd like the interface for each asset to be a little more compact (this is something I would do, for sure.)

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

The thought here is that if people create categories in a sane, structured way (which we can help with by seeding the db with some sane defaults), they can extend it out as needed - and we don't create models nearly as often as we create assets, so defining the assets at the category level, opting out of unnecessary ones at the model level, and then implementing them at the asset level seems like it corresponds well with the frequency of use for each of these levels of hierarchy.

@sigwo
Copy link

sigwo commented Dec 1, 2013

In my workplace, I need the ability to track multiple CATEGORIES with MAKES and several MODELS, LOCATIONS, and SERIAL NUMBERS/ASSET TAGS. As long as I can add/subtract fields for what I need, I would be happy. Example:

Network (Category)
Switch (Sub Category) - User created
Cisco (Make)
48 port (Custom Attribute) - User created
3750X (Model)
Serial Number
Location

I would like to see is a QR code/barcode generator to scan equipment for check out, scheduled inventories, accountability showdowns, etc. Enter information then click "Generate" to create/save/print code.

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

Hi @sigwo - barcodes are already on the planned feature list in #29.

I wonder if you couldn't accomplish the same thing with the structure we're talking about:

CATEGORY: Network: Switch

  • ports (Category Attribute)

MODELS:

  • Cisco 3750X
  • Cisco 3751X
  • Cisco 3752X
  • etc

@peelman
Copy link
Author

peelman commented Dec 1, 2013

So what bothers me about a lot of the discussion so far is that you are going to have a lot of categories and/or models that are essentially identical save for one or two distinguishing attributes. And unless we do a many-to-many and create a complicated custom attributes manager, you are going to have a lot of repeated custom attributes to handle all these common cases.

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

@peelman not sure I agree. Categories' basic purpose in this scenario is to be a collection of custom attributes. Can you give me use cases that support this notion?

@snipe
Copy link
Owner

snipe commented Dec 1, 2013

In @sigwo's example:

CATEGORY: Network:
SUB-CATEGORY: Switch

  • Num ports (Sub-Category Attribute)

MODELS:

Cisco 3750X
Cisco 3751X
Cisco 3752X
etc

So basically all switch models would have the same set of attributes available to them, since Switch is the (sub) category.

@sigwo
Copy link

sigwo commented Dec 1, 2013

@snipe Apologies for missing #29.

@peelman If I select Network -> Switch -> 3750X, I've already selected > 80% of my needed fields for completion. Basically, I would only need to fill in the Asset Tag/Serial Number and location. After the initial CATEGORY/SUB-CATEGORY setup, adding additional like-assets would speed up.

I'm not sure if you will be able to create the fields we want without creating a lot of fields we won't use.

@snipe
Copy link
Owner

snipe commented Dec 2, 2013

@sigwo HOW DARE YOU NOT KNOW EVERY OPEN GH ISSUE ON THIS PROJECT BY HEART! 👅

It's totally fine - just wanted to let you know that that that's in planning and is definitely part of what I consider the v1 MVP. In fact, I may ping you in that thread to clarify your use cases there. I don't need them, so your input would be helpful.

@tobint
Copy link

tobint commented Dec 2, 2013

I'm extremely exhausted at the moment, so if I'm thinking too general here

Every time I develop a system that is meant to be deployed by multiple unknown parties, the idea of customization/extensibility comes up. Inevitably, this same discussion comes up every time. The conclusion I always come to is: you can't be all things to all people and still have the best product for those customers. You really do have to make some choices that will limit your potential adopters or you will end up with a system so abstract that is either too complex or too slow for anyone to use.

Who's hero do you want to be? So before I can weigh in, I have to ask about your target audience. For instance, what size businesses? How many assets? How much customization do you want them to be able to make through administrative interface?

Understanding your target audience helps strike a balance between administrative configuration and module-based extensibility. Obviously, administrative configuration will lead you down the paths you've discussed above (and I'm happy to rehash the plusses minuses you've already pointed out), Module-based extensibility allows for ultimate optimization around data-type specifics. You can also create various hybrids of this. For instance, you can create a module-based system for extensibility and create some basic types up front that people can choose from and even extend through a certain number of fields -- leaving larger implementations and highly varying data types to plug in through modularity.

@snipe, you mentioned categories, and I think that's great. I wouldn't hard-code those decisions around those categories though. I'd use those as template starting points. In other words, if someone picks "laptop", they can then extend that with more fields. The problem is that no matter what you do, you're going to end up with too much abstraction to make the system cost-effective and fast. Think about this at even a field level and it can get extremely complex. Suppose you define data types such as a MAC address. You could then let the system know what kind of input mask to accept, client and server-side data validation, localized error message handling, data storage type, etc. You then put a bunch of these together and tell the system to both write to and query from this system on a regular basis for an asset type. It can get very slow if assets are swapping around a ton. More abstraction == more CPU/storage cost.

Sorry, I'm rambling, but my point is that if you define who your v1 customer is and the type of assets they want, then you can stay focused for now on those specifics.

For v2, you can create a more module-driven system with some built in module types (laptop, phone, switch, etc) and some "generic" modules that allow for configuration-based asset types. These types will be slower, but for smaller businesses without the dev resources to create a module, this shouldn't be a problem as they'll have less assets to track anyway. For those with more specific needs, building a module that allows them to search custom fields, validate data, provide custom error messages, etc -- they can build their own module for that.

Just my two cents for now. Sorry if this wasn't helpful. Will go to sleep and review how much damage I've done later. :)

@snipe
Copy link
Owner

snipe commented Dec 2, 2013

Thanks for the lecture on lean development, @tobint ;)

I don't disagree with anything you've said here - and I'm absolutely NOT trying to be all things to all people. I will not compromise my vision of what I want this to be to be everything to everyone, or even if it means the UX becomes so confusing and shitty that it's no better than every other crappy FOSS (and paid) tool out there.

That said, that's not a reason to think through extensibility. There will be no modules or anything like that, and there's not going to be a lot that's truly customizable through admin settings - this is simply a data architecture discussion.

Folks like Tracmor managed to pull off what I'm talking about here, and their code is shit, so there's no reason why we can't pull it off here either. The decision isn't whether to include custom fields - they ARE being included - the question is how to implement them in a way that makes the most sense based on the relationships of the fields to the data. While I'm a big fan of data normalization, I also have few qualms about de-norming for performance purposes, searchability, etc. This is more of a question of which property the fields get associated with. At this point, I think category still makes the most sense to me - and by changing the category, the user can extend it out as needed, based on the notion that a "category" is simply a collection of custom fields.

So I totally get it - and am just telling you not to worry so much :) While ambitious, I'm also exceptionally lazy and have no intention of over-engineering anything. Just giving some consideration right now. And we may get it all wrong and have to change it later. That's okay too :)

@tobint
Copy link

tobint commented Dec 2, 2013

Cool. Wasn't trying to lecture on lean dev. I know you get it. I felt actually silly at one point and deleted part of my post because I remembered who I was talking to. I'm so used to talking to those who have no idea how to break down the problem, let alone go from zero to MVP in a very short time.

@snipe
Copy link
Owner

snipe commented Dec 2, 2013

Nothing but love for you, @tobint - I know you meant it absolutely the right way - and I can get caught up in the details and get distracted from MVP, so it's a sound reminder. The difference in this project for me is that this MVP isn't for a company, it's for ME personally to make my life easier, so I'm hyper-focused :)

@peelman
Copy link
Author

peelman commented Dec 2, 2013

I had family to entertain last night for a belated thanksgiving, so I missed out on a lively discussion it seems 😄

My concern about duplicate custom attributes was under the assumption that custom attributes would be applied at the model level. As it stands, Categories seems like a second class citizen, a lookup table rather than an inheritance source. So if things can be applied at that level, that changes the ballgame somewhat. And @sigwo's example of subcategories is a completely new thing that hasn't been mentioned before (in anything I've read, at least).

So basically, based on what I've read, and how I interpret it, I want to say that while I see the value in a custom attributes system, building the entire foundation upon a collection of custom attributes and simply populating a default set seems like a terribly complicated way to go about things.

@rkayutkin rkayutkin mentioned this issue Jun 14, 2019
2 tasks
@swegaj
Copy link

swegaj commented Jan 21, 2020

I want create workstation in dropdown list in snipe-IT
Like Asset,User, License,
Please guide me on this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests